[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:09 dist-test-slave-znh6 krb5kdc[30946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:09 dist-test-slave-znh6 krb5kdc[30946](info): set up 2 sockets
Aug 08 07:10:09 dist-test-slave-znh6 krb5kdc[30946](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 07:10:10.779234 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:10 dist-test-slave-znh6 krb5kdc[30946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:10.897383 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:32817
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:32817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:10.976845 30965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:10.977072 30965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:10.977108 30965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:10.978192 30965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:10.978256 30965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:10.978281 30965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:10.978303 30965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:10.978329 30965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:10.979775 30965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:32817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:32817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:10.980031 30965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:10.980239 30965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:10.985081 30972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:10.985152 30971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:10.985297 30965 server_base.cc:1034] running on GCE node
W20260808 07:10:10.985088 30974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:10.985633 30965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:10.985890 30965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:10.987031 30965 hybrid_clock.cc:648] HybridClock initialized: now 1786173010987015 us; error 103 us; skew 500 ppm
Aug 08 07:10:10 dist-test-slave-znh6 krb5kdc[30946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:10.989382 30965 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:10.989738 30965 webserver.cc:460] Webserver started at http://127.30.54.126:35177/ using document root <none> and password file <none>
I20260808 07:10:10.989917 30965 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:10.989957 30965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:10.990043 30965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:10.991053 30965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "886d014dd5cd40d392aa0e301a7dc150"
format_stamp: "Formatted at 2026-08-08 07:10:10 on dist-test-slave-znh6"
server_key: "cb455d5e2669742c89002cde3153d7d6"
server_key_iv: "c183d51915c87c6488ed7d4b363e0e6e"
server_key_version: "encryptionkey@0"
I20260808 07:10:10.991354 30965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "886d014dd5cd40d392aa0e301a7dc150"
format_stamp: "Formatted at 2026-08-08 07:10:10 on dist-test-slave-znh6"
server_key: "cb455d5e2669742c89002cde3153d7d6"
server_key_iv: "c183d51915c87c6488ed7d4b363e0e6e"
server_key_version: "encryptionkey@0"
I20260808 07:10:10.992596 30965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:10.993258 30984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:10.993407 30965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:10.993489 30965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "886d014dd5cd40d392aa0e301a7dc150"
format_stamp: "Formatted at 2026-08-08 07:10:10 on dist-test-slave-znh6"
server_key: "cb455d5e2669742c89002cde3153d7d6"
server_key_iv: "c183d51915c87c6488ed7d4b363e0e6e"
server_key_version: "encryptionkey@0"
I20260808 07:10:10.993541 30965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:11.004485 30965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:11.006191 30965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:11.006294 30965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:11.010169 30965 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:32817
I20260808 07:10:11.010263 31054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:32817 every 8 connection(s)
I20260808 07:10:11.010553 30965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:11.011508 31055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:11.011691 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 30965
I20260808 07:10:11.011791 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:11.012426 30937 external_mini_cluster.cc:1428] Setting key e16f77740c435e06a32a06f41b79fdfc
I20260808 07:10:11.014245 31055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150: Bootstrap starting.
I20260808 07:10:11.014878 31055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:11.015188 31055 log.cc:824] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:11.016000 31055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150: No bootstrap required, opened a new log
I20260808 07:10:11.017807 31055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "886d014dd5cd40d392aa0e301a7dc150" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32817 } }
I20260808 07:10:11.017963 31055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:11.018021 31055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 886d014dd5cd40d392aa0e301a7dc150, State: Initialized, Role: FOLLOWER
I20260808 07:10:11.018157 31055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "886d014dd5cd40d392aa0e301a7dc150" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32817 } }
I20260808 07:10:11.018237 31055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:11.018280 31055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:11.018328 31055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:11.018922 31055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "886d014dd5cd40d392aa0e301a7dc150" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32817 } }
I20260808 07:10:11.019037 31055 leader_election.cc:302] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 886d014dd5cd40d392aa0e301a7dc150; no voters:
I20260808 07:10:11.019225 31055 leader_election.cc:288] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:11.019253 31060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:11.019376 31060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [term 1 LEADER]: Becoming Leader. State: Replica: 886d014dd5cd40d392aa0e301a7dc150, State: Running, Role: LEADER
I20260808 07:10:11.019500 31055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:11.019485 31060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "886d014dd5cd40d392aa0e301a7dc150" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32817 } }
I20260808 07:10:11.019846 31060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 886d014dd5cd40d392aa0e301a7dc150. Latest consensus state: current_term: 1 leader_uuid: "886d014dd5cd40d392aa0e301a7dc150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "886d014dd5cd40d392aa0e301a7dc150" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32817 } } }
I20260808 07:10:11.019934 31060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:11.019867 31062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "886d014dd5cd40d392aa0e301a7dc150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "886d014dd5cd40d392aa0e301a7dc150" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32817 } } }
I20260808 07:10:11.020083 31062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:11.020179 31068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:11.020888 31068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:11.024263 31068 catalog_manager.cc:1347] Generated new cluster ID: 0056ac7dd29a4b718b2e4515626a5ef3
I20260808 07:10:11.024324 31068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:11.028450 31068 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:11.029137 31068 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:11.029953 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.013406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37536 (local address 127.30.54.126:32817)
0808 07:10:11.013576 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:11.013585 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:11.013835 (+ 250us) server_negotiation.cc:407] Connection header received
0808 07:10:11.016162 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:11.016167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:11.016211 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:11.016347 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:11.017792 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.018922 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.020413 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.021170 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.021314 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:11.025748 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:11.025752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:11.025752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:11.025764 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:11.027709 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.028282 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.028285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.028286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.028389 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.028703 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.028704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.028705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.029015 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:11.029172 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:11.029614 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 07:10:11.029754 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260808 07:10:11.034584 31068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 886d014dd5cd40d392aa0e301a7dc150: Generated new TSK 0
I20260808 07:10:11.034716 31068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:10:11.061127 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32817
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:11.142129 31095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:11.142347 31095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:11.142441 31095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:11.143646 31095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:11.143707 31095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:11.143751 31095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:11.145179 31095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32817
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:11.145440 31095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:11.145673 31095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:11.150597 31101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:11.150619 31104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:11.150631 31095 server_base.cc:1034] running on GCE node
W20260808 07:10:11.150597 31108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:11.151002 31095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:11.151278 31095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:11.152462 31095 hybrid_clock.cc:648] HybridClock initialized: now 1786173011152441 us; error 30 us; skew 500 ppm
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:11.155015 31095 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:10:11.155434 31095 webserver.cc:460] Webserver started at http://127.30.54.65:42627/ using document root <none> and password file <none>
I20260808 07:10:11.155691 31095 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:11.155761 31095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:11.155910 31095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:11.156898 31095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "dadd973be4ef4b89a06e7b96be0a2e9b"
format_stamp: "Formatted at 2026-08-08 07:10:11 on dist-test-slave-znh6"
server_key: "dce403a83774fed64714a8ff2c13f8e6"
server_key_iv: "7c055fe687d45b2efa3c62028e398378"
server_key_version: "encryptionkey@0"
I20260808 07:10:11.157248 31095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "dadd973be4ef4b89a06e7b96be0a2e9b"
format_stamp: "Formatted at 2026-08-08 07:10:11 on dist-test-slave-znh6"
server_key: "dce403a83774fed64714a8ff2c13f8e6"
server_key_iv: "7c055fe687d45b2efa3c62028e398378"
server_key_version: "encryptionkey@0"
I20260808 07:10:11.158746 31095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:11.159487 31116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.159653 31095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.159752 31095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "dadd973be4ef4b89a06e7b96be0a2e9b"
format_stamp: "Formatted at 2026-08-08 07:10:11 on dist-test-slave-znh6"
server_key: "dce403a83774fed64714a8ff2c13f8e6"
server_key_iv: "7c055fe687d45b2efa3c62028e398378"
server_key_version: "encryptionkey@0"
I20260808 07:10:11.159825 31095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:11.167894 31095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:11.169543 31095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:11.169691 31095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:11.169899 31095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:11.170185 31095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:11.170236 31095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.170284 31095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:11.170311 31095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.176379 31095 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:38239
I20260808 07:10:11.176445 31270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:38239 every 8 connection(s)
I20260808 07:10:11.176743 31095 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:11.185139 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.177445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:40193 (local address 127.30.54.126:32817)
0808 07:10:11.177499 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:11.177501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:11.177681 (+ 180us) server_negotiation.cc:407] Connection header received
0808 07:10:11.178119 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:11.178120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:11.178129 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:11.178171 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:11.179148 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.179652 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.180620 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.180808 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.180862 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:11.182757 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:11.182759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:11.182759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:11.182766 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:11.184057 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.184419 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.184419 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.184420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.184468 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.184665 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.184666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.184666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.184759 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:11.184808 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:11.185014 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 07:10:11.185076 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:10:11.185376 31273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.177416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:32817 (local address 127.30.54.65:40193)
0808 07:10:11.177563 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:11.177601 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:11.178028 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:11.178219 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:11.178222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:11.178887 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:11.179072 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:11.179076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.179733 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.179734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:11.180542 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:11.180544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.180878 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.180879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:11.181011 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:11.181394 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:11.181402 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:11.182676 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:11.184127 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:11.184133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:11.184135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:11.184356 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:11.184523 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:11.184524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:11.184524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:11.184615 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:11.184810 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:11.184814 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:11.184913 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 07:10:11.184991 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:10:11.185098 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260808 07:10:11.185878 31271 heartbeater.cc:347] Connected to a master server at 127.30.54.126:32817
I20260808 07:10:11.185981 31271 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:11.186020 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31095
I20260808 07:10:11.186113 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:11.186194 31271 heartbeater.cc:511] Master 127.30.54.126:32817 requested a full tablet report, sending...
I20260808 07:10:11.186276 30937 external_mini_cluster.cc:1428] Setting key f6ce29821d5ed4fc6d3e82d50639d2cc
I20260808 07:10:11.186780 31006 ts_manager.cc:194] Registered new tserver with Master: dadd973be4ef4b89a06e7b96be0a2e9b (127.30.54.65:38239)
I20260808 07:10:11.187674 31006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:40193
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:10:11.211445 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32817
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:11.292644 31278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:11.292832 31278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:11.292855 31278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:11.293952 31278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:11.293992 31278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:11.294018 31278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:11.295557 31278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32817
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:11.295765 31278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:11.295987 31278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:11.300943 31289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:11.300980 31284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:11.300978 31286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:11.301158 31278 server_base.cc:1034] running on GCE node
I20260808 07:10:11.301349 31278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:11.301537 31278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:11.302696 31278 hybrid_clock.cc:648] HybridClock initialized: now 1786173011302681 us; error 28 us; skew 500 ppm
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:11.305289 31278 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:10:11.305763 31278 webserver.cc:460] Webserver started at http://127.30.54.66:40211/ using document root <none> and password file <none>
I20260808 07:10:11.305977 31278 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:11.306043 31278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:11.306188 31278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:11.307257 31278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "54a4d6ef4cf14357b33162ccfaa3074b"
format_stamp: "Formatted at 2026-08-08 07:10:11 on dist-test-slave-znh6"
server_key: "53dfe2583fa36275f1d106b310daa1cf"
server_key_iv: "360b0436cdab83e9c914ab14fdc62e48"
server_key_version: "encryptionkey@0"
I20260808 07:10:11.307611 31278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "54a4d6ef4cf14357b33162ccfaa3074b"
format_stamp: "Formatted at 2026-08-08 07:10:11 on dist-test-slave-znh6"
server_key: "53dfe2583fa36275f1d106b310daa1cf"
server_key_iv: "360b0436cdab83e9c914ab14fdc62e48"
server_key_version: "encryptionkey@0"
I20260808 07:10:11.308928 31278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:11.309690 31298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.309852 31278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:11.309952 31278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "54a4d6ef4cf14357b33162ccfaa3074b"
format_stamp: "Formatted at 2026-08-08 07:10:11 on dist-test-slave-znh6"
server_key: "53dfe2583fa36275f1d106b310daa1cf"
server_key_iv: "360b0436cdab83e9c914ab14fdc62e48"
server_key_version: "encryptionkey@0"
I20260808 07:10:11.310020 31278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:11.321620 31278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:11.323822 31278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:11.323982 31278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:11.324204 31278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:11.324527 31278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:11.324580 31278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.324620 31278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:11.324647 31278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.330762 31278 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:34359
I20260808 07:10:11.330838 31455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:34359 every 8 connection(s)
I20260808 07:10:11.331117 31278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:11.335994 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31278
I20260808 07:10:11.336072 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:11.336263 30937 external_mini_cluster.cc:1428] Setting key 79f5c8721589485fdbfb2c993af08be5
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:11.340540 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.331790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:57667 (local address 127.30.54.126:32817)
0808 07:10:11.331835 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:11.331837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:11.332020 (+ 183us) server_negotiation.cc:407] Connection header received
0808 07:10:11.332458 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:11.332459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:11.332469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:11.332525 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:11.333355 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.333845 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.334792 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.335051 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.335106 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:11.337237 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:11.337238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:11.337239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:11.337247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:11.339024 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.339518 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.339519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.339519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.339572 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.339868 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.339869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.339869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.339985 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:11.340051 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:11.340359 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 07:10:11.340449 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:11.340703 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.331758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:32817 (local address 127.30.54.66:57667)
0808 07:10:11.331903 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:11.331935 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:11.332379 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:11.332572 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:11.332575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:11.333111 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:11.333283 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:11.333287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.333930 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.333930 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:11.334721 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:11.334723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.335121 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.335122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:11.335219 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:11.335608 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:11.335615 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:11.337147 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:11.339100 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:11.339108 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:11.339110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:11.339426 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:11.339640 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:11.339641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:11.339641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:11.339773 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:11.340058 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:11.340063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:11.340203 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:10:11.340319 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 07:10:11.340476 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260808 07:10:11.341179 31457 heartbeater.cc:347] Connected to a master server at 127.30.54.126:32817
I20260808 07:10:11.341267 31457 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:11.341441 31457 heartbeater.cc:511] Master 127.30.54.126:32817 requested a full tablet report, sending...
I20260808 07:10:11.341851 31006 ts_manager.cc:194] Registered new tserver with Master: 54a4d6ef4cf14357b33162ccfaa3074b (127.30.54.66:34359)
I20260808 07:10:11.342396 31006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:57667
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:10:11.363727 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32817
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:11.445544 31465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:11.445735 31465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:11.445760 31465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:11.446887 31465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:11.446928 31465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:11.446977 31465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:11.448417 31465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32817
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:11.448650 31465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:11.448868 31465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:11.454006 31472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:11.454043 31465 server_base.cc:1034] running on GCE node
W20260808 07:10:11.454015 31473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:11.454015 31475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:11.454517 31465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:11.454761 31465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:11.455922 31465 hybrid_clock.cc:648] HybridClock initialized: now 1786173011455902 us; error 36 us; skew 500 ppm
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:11.458339 31465 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:10:11.458806 31465 webserver.cc:460] Webserver started at http://127.30.54.67:38129/ using document root <none> and password file <none>
I20260808 07:10:11.459097 31465 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:11.459142 31465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:11.459287 31465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:11.460261 31465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "005525473ef14a28b5418042ffc646f5"
format_stamp: "Formatted at 2026-08-08 07:10:11 on dist-test-slave-znh6"
server_key: "1adcb336343c95980a341d771958ec7a"
server_key_iv: "ab2c612e6ca3c004ceaa29309ac57045"
server_key_version: "encryptionkey@0"
I20260808 07:10:11.460613 31465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "005525473ef14a28b5418042ffc646f5"
format_stamp: "Formatted at 2026-08-08 07:10:11 on dist-test-slave-znh6"
server_key: "1adcb336343c95980a341d771958ec7a"
server_key_iv: "ab2c612e6ca3c004ceaa29309ac57045"
server_key_version: "encryptionkey@0"
I20260808 07:10:11.461930 31465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:11.462653 31483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.462886 31465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:11.462980 31465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "005525473ef14a28b5418042ffc646f5"
format_stamp: "Formatted at 2026-08-08 07:10:11 on dist-test-slave-znh6"
server_key: "1adcb336343c95980a341d771958ec7a"
server_key_iv: "ab2c612e6ca3c004ceaa29309ac57045"
server_key_version: "encryptionkey@0"
I20260808 07:10:11.463057 31465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:11.490218 31465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:11.491968 31465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:11.492125 31465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:11.492357 31465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:11.492677 31465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:11.492731 31465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.492766 31465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:11.492794 31465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:11.499361 31465 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:36155
I20260808 07:10:11.499397 31639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:36155 every 8 connection(s)
I20260808 07:10:11.499715 31465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:11.508234 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.500433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:56791 (local address 127.30.54.126:32817)
0808 07:10:11.500493 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:11.500495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:11.500703 (+ 208us) server_negotiation.cc:407] Connection header received
0808 07:10:11.501145 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:11.501146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:11.501156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:11.501212 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:11.502050 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.502550 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.503530 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.503714 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.503768 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:11.505914 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:11.505915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:11.505916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:11.505922 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:11.507210 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.507561 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.507562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.507562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.507610 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.507772 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.507772 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.507773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.507875 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:11.507920 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:11.508114 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 07:10:11.508172 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:10:11.508376 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.500416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:32817 (local address 127.30.54.67:56791)
0808 07:10:11.500582 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:11.500620 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:11.501068 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:11.501260 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:11.501263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:11.501802 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:11.501978 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:11.501983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.502625 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.502626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:11.503452 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:11.503455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.503782 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.503782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:11.503874 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:11.504375 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:11.504383 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:11.505812 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:11.507279 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:11.507285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:11.507286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:11.507502 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:11.507650 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:11.507650 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:11.507650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:11.507727 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:11.507919 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:11.507921 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:11.508020 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 07:10:11.508092 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:10:11.508198 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260808 07:10:11.508904 31640 heartbeater.cc:347] Connected to a master server at 127.30.54.126:32817
I20260808 07:10:11.508919 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31465
I20260808 07:10:11.508988 31640 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:11.509017 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:11.509167 31640 heartbeater.cc:511] Master 127.30.54.126:32817 requested a full tablet report, sending...
I20260808 07:10:11.509186 30937 external_mini_cluster.cc:1428] Setting key 30f6991c1e16bfb2201e375d3372c650
I20260808 07:10:11.509563 31006 ts_manager.cc:194] Registered new tserver with Master: 005525473ef14a28b5418042ffc646f5 (127.30.54.67:36155)
I20260808 07:10:11.510061 31006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:56791
I20260808 07:10:11.510258 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:11.534198 31650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.526589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.30.54.65:38239)
0808 07:10:11.526774 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:11.526778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:11.526787 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:10:11.526810 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:11.526855 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:11.526872 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:11.526958 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:11.527527 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.528247 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.528977 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.529286 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.529374 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:11.531553 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:11.531561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:11.531562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:11.531569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:11.533014 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.533388 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.533391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.533392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.533458 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.533659 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.533660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.533660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.533795 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:11.533947 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:11.533996 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 07:10:11.534074 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
W20260808 07:10:11.534585 31154 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36882
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[30946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:11.557003 31650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.550149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.30.54.65:38239)
0808 07:10:11.550208 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:11.550210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:11.550286 (+ 76us) server_negotiation.cc:407] Connection header received
0808 07:10:11.550368 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:11.550369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:11.550384 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:11.550442 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:11.551031 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.551513 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.552246 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:11.552458 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:11.552534 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:11.554529 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:11.554531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:11.554531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:11.554538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:11.555880 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.556257 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.556258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.556259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.556314 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:11.556510 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:11.556511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:11.556511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:11.556607 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:11.556732 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:11.556823 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 07:10:11.556902 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:10:11.557464 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31095
I20260808 07:10:11.566659 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31278
I20260808 07:10:11.575368 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31465
I20260808 07:10:11.584913 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 30965
2026-08-08T07:10:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2080 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[31659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[31659](info): set up 2 sockets
Aug 08 07:10:11 dist-test-slave-znh6 krb5kdc[31659](info): commencing operation
krb5kdc: starting...
W20260808 07:10:12.802459 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:12 dist-test-slave-znh6 krb5kdc[31659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:12.920010 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:32845
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:32845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:13.001250 31681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:13.001442 31681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:13.001470 31681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:13.002601 31681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:13.002653 31681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:13.002667 31681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:13.002700 31681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:13.002727 31681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:13.004312 31681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:32845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:32845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:13.004562 31681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:13.004781 31681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:13.009541 31686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:13.009552 31690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:13.009663 31681 server_base.cc:1034] running on GCE node
W20260808 07:10:13.009567 31688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:13.010035 31681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:13.010370 31681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:13.011524 31681 hybrid_clock.cc:648] HybridClock initialized: now 1786173013011503 us; error 34 us; skew 500 ppm
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:13.014461 31681 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:13.014925 31681 webserver.cc:460] Webserver started at http://127.30.54.126:45229/ using document root <none> and password file <none>
I20260808 07:10:13.015139 31681 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:13.015204 31681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:13.015341 31681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:13.016337 31681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "9b334541ab257b760e50344f3888cdba"
server_key_iv: "6171133750b264b9c524b4dbb5a34e9c"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.016688 31681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "9b334541ab257b760e50344f3888cdba"
server_key_iv: "6171133750b264b9c524b4dbb5a34e9c"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.017966 31681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:13.018672 31704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.018863 31681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:13.018966 31681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "9b334541ab257b760e50344f3888cdba"
server_key_iv: "6171133750b264b9c524b4dbb5a34e9c"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.019047 31681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:13.040238 31681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:13.042076 31681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:13.042228 31681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:13.046906 31681 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:32845
I20260808 07:10:13.046959 31774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:32845 every 8 connection(s)
I20260808 07:10:13.047362 31681 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:13.048199 31775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:13.050354 31775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b: Bootstrap starting.
I20260808 07:10:13.050937 31775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:13.051183 31775 log.cc:824] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:13.051816 31775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b: No bootstrap required, opened a new log
I20260808 07:10:13.053310 31775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32845 } }
I20260808 07:10:13.053440 31775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:13.053463 31775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac0fda1e1db4aaa8ba497eb007c5b8b, State: Initialized, Role: FOLLOWER
I20260808 07:10:13.053597 31775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32845 } }
I20260808 07:10:13.053681 31775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:13.053704 31775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:13.053738 31775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:13.054217 31775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32845 } }
I20260808 07:10:13.054306 31775 leader_election.cc:302] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ac0fda1e1db4aaa8ba497eb007c5b8b; no voters:
I20260808 07:10:13.054456 31775 leader_election.cc:288] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:13.054508 31780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:13.054581 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31681
I20260808 07:10:13.054669 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:13.054688 31780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [term 1 LEADER]: Becoming Leader. State: Replica: 3ac0fda1e1db4aaa8ba497eb007c5b8b, State: Running, Role: LEADER
I20260808 07:10:13.054708 31775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:13.054771 31780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32845 } }
I20260808 07:10:13.054855 30937 external_mini_cluster.cc:1428] Setting key b1196f6b810f515c247a1e6512a2e790
I20260808 07:10:13.055308 31781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32845 } } }
I20260808 07:10:13.055404 31781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:13.055707 31782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ac0fda1e1db4aaa8ba497eb007c5b8b. Latest consensus state: current_term: 1 leader_uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac0fda1e1db4aaa8ba497eb007c5b8b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 32845 } } }
I20260808 07:10:13.055861 31782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:13.056069 31793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:13.056506 31793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:13.058233 31793 catalog_manager.cc:1347] Generated new cluster ID: 98302915378842fe9853e13326559929
I20260808 07:10:13.058293 31793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:13.065459 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.055468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49472 (local address 127.30.54.126:32845)
0808 07:10:13.055826 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.055850 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.055883 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:10:13.056019 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.056027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.056094 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.057254 (+ 1160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:13.057333 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.058567 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.059393 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.059827 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.059945 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.062218 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:13.062222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:13.062222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:13.062234 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:13.063942 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.064330 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.064333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.064334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.064435 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.064651 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.064652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.064653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.064889 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:13.065089 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.065155 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.065292 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":79,"threads_started":1}
I20260808 07:10:13.065989 31793 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:13.066645 31793 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:13.077278 31793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ac0fda1e1db4aaa8ba497eb007c5b8b: Generated new TSK 0
I20260808 07:10:13.077435 31793 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:10:13.113484 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32845
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:13.210148 31812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:13.210347 31812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:13.210374 31812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:13.211568 31812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:13.211608 31812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:13.211634 31812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:13.213093 31812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32845
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:13.213328 31812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:13.213550 31812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:13.218477 31821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:13.218464 31823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:13.218468 31820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:13.218745 31812 server_base.cc:1034] running on GCE node
I20260808 07:10:13.218968 31812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:13.219231 31812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:13.220402 31812 hybrid_clock.cc:648] HybridClock initialized: now 1786173013220391 us; error 38 us; skew 500 ppm
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:13.223088 31812 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:10:13.223476 31812 webserver.cc:460] Webserver started at http://127.30.54.65:46197/ using document root <none> and password file <none>
I20260808 07:10:13.223657 31812 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:13.223697 31812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:13.223783 31812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:13.224748 31812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "332c94d2831e48ff8b45ff45a62e2921"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "10bcdfb4ef8de1e95472a281f4c24b25"
server_key_iv: "92dd1d76aadcf45d8f8c713cfabddacc"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.225067 31812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "332c94d2831e48ff8b45ff45a62e2921"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "10bcdfb4ef8de1e95472a281f4c24b25"
server_key_iv: "92dd1d76aadcf45d8f8c713cfabddacc"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.226326 31812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:13.227083 31831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.227275 31812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:13.227371 31812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "332c94d2831e48ff8b45ff45a62e2921"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "10bcdfb4ef8de1e95472a281f4c24b25"
server_key_iv: "92dd1d76aadcf45d8f8c713cfabddacc"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.227433 31812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:13.235107 31812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:13.236925 31812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:13.237031 31812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:13.237197 31812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:13.237448 31812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:13.237489 31812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.237509 31812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:13.237521 31812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.243506 31812 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:35421
I20260808 07:10:13.243602 31988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:35421 every 8 connection(s)
I20260808 07:10:13.243897 31812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:13.248907 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31812
I20260808 07:10:13.249028 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:13.249199 30937 external_mini_cluster.cc:1428] Setting key 3a96f59ec5a7cbc37e5888abdee8610f
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:13.253830 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.244644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:53867 (local address 127.30.54.126:32845)
0808 07:10:13.244712 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.244713 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.244922 (+ 209us) server_negotiation.cc:407] Connection header received
0808 07:10:13.245810 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.245811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.245821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.245880 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:13.246469 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.247058 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.248084 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.248304 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.248387 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.250710 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:13.250712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:13.250712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:13.250721 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:13.252442 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.252842 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.252843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.252844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.252907 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.253168 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.253169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.253170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.253302 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:13.253371 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.253644 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.253730 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:10:13.254006 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.244605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:32845 (local address 127.30.54.65:53867)
0808 07:10:13.244800 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.244841 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:13.245711 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:13.245939 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:13.245942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:13.246173 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:13.246383 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.246387 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.247141 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.247142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.247972 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.247974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.248420 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.248421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.248533 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.249040 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:13.249052 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:13.250612 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:13.252542 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:13.252549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:13.252550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:13.252770 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:13.252989 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:13.252990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:13.252991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:13.253099 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:13.253374 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:13.253382 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:13.253513 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 07:10:13.253619 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:10:13.253773 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260808 07:10:13.254751 31989 heartbeater.cc:347] Connected to a master server at 127.30.54.126:32845
I20260808 07:10:13.254886 31989 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:13.255069 31989 heartbeater.cc:511] Master 127.30.54.126:32845 requested a full tablet report, sending...
I20260808 07:10:13.255610 31727 ts_manager.cc:194] Registered new tserver with Master: 332c94d2831e48ff8b45ff45a62e2921 (127.30.54.65:35421)
I20260808 07:10:13.256242 31727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:53867
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:10:13.275771 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32845
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:13.356768 31994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:13.356966 31994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:13.356988 31994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:13.358664 31994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:13.358706 31994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:13.358733 31994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:13.360229 31994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32845
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:13.360481 31994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:13.360707 31994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:13.365610 32001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:13.365645 32004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:13.365871 32002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:13.365886 31994 server_base.cc:1034] running on GCE node
I20260808 07:10:13.366146 31994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:13.366395 31994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:13.367538 31994 hybrid_clock.cc:648] HybridClock initialized: now 1786173013367522 us; error 33 us; skew 500 ppm
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:13.370059 31994 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:10:13.370494 31994 webserver.cc:460] Webserver started at http://127.30.54.66:42705/ using document root <none> and password file <none>
I20260808 07:10:13.370718 31994 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:13.370784 31994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:13.370977 31994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:13.371989 31994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "473db6eb65cf417d941fb3f58c758516"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "089d79cee30d469fedb7fe1b9dcfcf17"
server_key_iv: "3d485552cb9d45ddd6c371176575b8f5"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.372359 31994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "473db6eb65cf417d941fb3f58c758516"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "089d79cee30d469fedb7fe1b9dcfcf17"
server_key_iv: "3d485552cb9d45ddd6c371176575b8f5"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.373757 31994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:13.374462 32012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.374670 31994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:13.374770 31994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "473db6eb65cf417d941fb3f58c758516"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "089d79cee30d469fedb7fe1b9dcfcf17"
server_key_iv: "3d485552cb9d45ddd6c371176575b8f5"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.374878 31994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:13.384622 31994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:13.386417 31994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:13.386574 31994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:13.386786 31994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:13.387152 31994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:13.387205 31994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.387245 31994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:13.387274 31994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.392993 31994 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:37887
I20260808 07:10:13.393043 32172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:37887 every 8 connection(s)
I20260808 07:10:13.393337 31994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:13.400414 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31994
I20260808 07:10:13.400513 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:13.400995 30937 external_mini_cluster.cc:1428] Setting key 22b753e4c9276cb5c79dd431b7e5e53d
I20260808 07:10:13.402871 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.394001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:47097 (local address 127.30.54.126:32845)
0808 07:10:13.394071 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.394073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.394289 (+ 216us) server_negotiation.cc:407] Connection header received
0808 07:10:13.395239 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.395241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.395252 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.395308 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:13.395880 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.396396 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.397352 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.397540 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.397594 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.399745 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:13.399746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:13.399747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:13.399754 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:13.401441 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.401822 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.401823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.401824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.401890 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.402116 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.402117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.402117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.402264 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:13.402337 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.402614 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.402713 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:10:13.402941 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.393969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:32845 (local address 127.30.54.66:47097)
0808 07:10:13.394144 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.394179 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:13.395137 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:13.395362 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:13.395365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:13.395616 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:13.395793 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.395797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.396472 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.396473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.397258 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.397260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.397634 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.397635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.397772 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.398268 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:13.398279 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:13.399661 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:13.401520 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:13.401526 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:13.401528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:13.401751 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:13.401960 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:13.401960 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:13.401961 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:13.402054 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:13.402347 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:13.402353 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:13.402480 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 07:10:13.402579 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 07:10:13.402703 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260808 07:10:13.403720 32173 heartbeater.cc:347] Connected to a master server at 127.30.54.126:32845
I20260808 07:10:13.403811 32173 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:13.404002 32173 heartbeater.cc:511] Master 127.30.54.126:32845 requested a full tablet report, sending...
I20260808 07:10:13.404361 31727 ts_manager.cc:194] Registered new tserver with Master: 473db6eb65cf417d941fb3f58c758516 (127.30.54.66:37887)
I20260808 07:10:13.404839 31727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:47097
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:10:13.430053 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32845
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:13.511719 32178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:13.511933 32178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:13.511968 32178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:13.513072 32178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:13.513130 32178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:13.513175 32178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:13.514668 32178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:32845
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:13.514948 32178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:13.515161 32178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:10:13.520025 32178 server_base.cc:1034] running on GCE node
W20260808 07:10:13.520041 32188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:13.520072 32189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:13.520131 32194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:13.520396 32178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:13.520649 32178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:13.521786 32178 hybrid_clock.cc:648] HybridClock initialized: now 1786173013521779 us; error 31 us; skew 500 ppm
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:13.524340 32178 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:10:13.524785 32178 webserver.cc:460] Webserver started at http://127.30.54.67:36449/ using document root <none> and password file <none>
I20260808 07:10:13.524999 32178 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:13.525066 32178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:13.525208 32178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:13.526180 32178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "b3cfbbe6082349aa8288bf69eb44f421"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "7a61ebc1881a99c50fe35ebf41414936"
server_key_iv: "6e9be97c085a6de3260be23893108e45"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.526548 32178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "b3cfbbe6082349aa8288bf69eb44f421"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "7a61ebc1881a99c50fe35ebf41414936"
server_key_iv: "6e9be97c085a6de3260be23893108e45"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.527915 32178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:13.528610 32204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.528769 32178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:13.528861 32178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "b3cfbbe6082349aa8288bf69eb44f421"
format_stamp: "Formatted at 2026-08-08 07:10:13 on dist-test-slave-znh6"
server_key: "7a61ebc1881a99c50fe35ebf41414936"
server_key_iv: "6e9be97c085a6de3260be23893108e45"
server_key_version: "encryptionkey@0"
I20260808 07:10:13.528932 32178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:13.555338 32178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:13.557334 32178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:13.557495 32178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:13.557711 32178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:13.558018 32178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:13.558072 32178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.558113 32178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:13.558141 32178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:13.564010 32178 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:38405
I20260808 07:10:13.564059 32364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:38405 every 8 connection(s)
I20260808 07:10:13.564366 32178 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:13.565302 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 32178
I20260808 07:10:13.565428 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:13.565598 30937 external_mini_cluster.cc:1428] Setting key 504bc1eba230b3ef25c974956b6b631c
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:13.573678 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.564970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:50811 (local address 127.30.54.126:32845)
0808 07:10:13.565035 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.565037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.565247 (+ 210us) server_negotiation.cc:407] Connection header received
0808 07:10:13.566549 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.566550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.566564 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.566614 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:13.567253 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.567764 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.568756 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.568976 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.569049 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.571105 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:13.571107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:13.571107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:13.571114 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:13.572292 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.572716 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.572717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.572717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.572779 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.573026 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.573027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.573027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.573146 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:13.573216 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.573504 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.573589 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:10:13.573854 32368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.564973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:32845 (local address 127.30.54.67:50811)
0808 07:10:13.565135 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.565169 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:13.566440 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:13.566678 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:13.566682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:13.566988 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:13.567164 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.567168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.567855 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.567856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.568664 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.568666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.569059 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.569060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.569191 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.569591 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:13.569599 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:13.570986 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:13.572407 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:13.572414 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:13.572416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:13.572636 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:13.572848 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:13.572849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:13.572849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:13.572959 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:13.573219 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:13.573225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:13.573363 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 07:10:13.573467 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 07:10:13.573616 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260808 07:10:13.574316 32366 heartbeater.cc:347] Connected to a master server at 127.30.54.126:32845
I20260808 07:10:13.574410 32366 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:13.574545 32366 heartbeater.cc:511] Master 127.30.54.126:32845 requested a full tablet report, sending...
I20260808 07:10:13.574949 31727 ts_manager.cc:194] Registered new tserver with Master: b3cfbbe6082349aa8288bf69eb44f421 (127.30.54.67:38405)
I20260808 07:10:13.575274 31727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:50811
I20260808 07:10:13.577215 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:13.584524 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.578854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49488 (local address 127.30.54.126:32845)
0808 07:10:13.578907 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.578909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.579004 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:10:13.579056 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.579057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.579066 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.579112 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:13.579603 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.580093 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.580756 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.581066 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.581116 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.582249 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:13.582250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:13.582251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:13.582256 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:13.583453 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.583771 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.583773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.583773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.583829 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.584034 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.584035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.584035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.584170 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:13.584246 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.584332 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.584396 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:10:13.586633 31727 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:13.588277 31727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:13.594635 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.590516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:35421 (local address 127.0.0.1:37538)
0808 07:10:13.590657 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.590686 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:13.590775 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:13.591192 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:13.591195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:13.591196 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:13.591275 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.591277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.592378 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.592379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.593543 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.593545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.594345 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.594346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.594424 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.594437 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:10:13.594506 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:10:13.594560 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260808 07:10:13.594648 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.590656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.30.54.65:35421)
0808 07:10:13.590909 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.590915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.590926 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:13.590981 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.590983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.591001 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.591091 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:13.591363 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.592279 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.593643 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.594260 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.594345 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.594404 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.594529 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.594571 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
I20260808 07:10:13.595055 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.590936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:38405 (local address 127.0.0.1:55226)
0808 07:10:13.591441 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.591456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:13.591521 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:13.591796 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:13.591797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:13.591799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:13.591837 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.591838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.592983 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.592985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.593897 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.593900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.594808 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.594809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.594900 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.594904 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:13.594937 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 07:10:13.594978 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":72,"threads_started":1}
I20260808 07:10:13.595245 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.590740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:37887 (local address 127.0.0.1:41636)
0808 07:10:13.591059 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.591069 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:13.591112 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:13.591285 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:13.591286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:13.591288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:13.591334 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.591335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.592811 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.592812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.593875 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.593877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.594902 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.594903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.594959 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.594962 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:13.595007 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:10:13.595038 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":67,"threads_started":1}
I20260808 07:10:13.595237 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.590909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41636 (local address 127.30.54.66:37887)
0808 07:10:13.591114 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.591119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.591132 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:10:13.591174 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.591176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.591190 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.591277 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:13.591527 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.592686 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.593980 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.594701 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.594776 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.594863 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.595096 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.595151 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
I20260808 07:10:13.595738 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.591032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.30.54.67:38405)
0808 07:10:13.591650 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.591654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.591663 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:10:13.591698 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.591700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.591715 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.591797 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:13.591918 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.592891 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.594004 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.594701 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.595551 (+ 850us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.595596 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.595616 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.595647 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":79,"threads_started":1}
I20260808 07:10:13.595695 31895 tablet_service.cc:1467] Processing CreateTablet for tablet 912e535a382447dc92b720f4a0d87cb9 (DEFAULT_TABLE table=test-table [id=a93bee7eb84f4ca38a7776d16dd492b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:13.595889 32081 tablet_service.cc:1467] Processing CreateTablet for tablet 912e535a382447dc92b720f4a0d87cb9 (DEFAULT_TABLE table=test-table [id=a93bee7eb84f4ca38a7776d16dd492b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:13.596067 31895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 912e535a382447dc92b720f4a0d87cb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:13.596184 32081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 912e535a382447dc92b720f4a0d87cb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:13.599251 32262 tablet_service.cc:1467] Processing CreateTablet for tablet 912e535a382447dc92b720f4a0d87cb9 (DEFAULT_TABLE table=test-table [id=a93bee7eb84f4ca38a7776d16dd492b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:13.599475 32392 tablet_bootstrap.cc:492] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516: Bootstrap starting.
I20260808 07:10:13.599568 32262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 912e535a382447dc92b720f4a0d87cb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:13.600128 32392 tablet_bootstrap.cc:654] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:13.600726 32392 log.cc:824] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:13.601094 32389 tablet_bootstrap.cc:492] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921: Bootstrap starting.
I20260808 07:10:13.601469 32392 tablet_bootstrap.cc:492] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516: No bootstrap required, opened a new log
I20260808 07:10:13.601516 32392 ts_tablet_manager.cc:1397] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:13.601751 32389 tablet_bootstrap.cc:654] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:13.602324 32389 log.cc:824] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:13.603183 32389 tablet_bootstrap.cc:492] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921: No bootstrap required, opened a new log
I20260808 07:10:13.603274 32389 ts_tablet_manager.cc:1397] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:13.603346 32397 tablet_bootstrap.cc:492] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421: Bootstrap starting.
I20260808 07:10:13.603439 32392 raft_consensus.cc:348] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } }
I20260808 07:10:13.603612 32392 raft_consensus.cc:374] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:13.603657 32392 raft_consensus.cc:729] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 473db6eb65cf417d941fb3f58c758516, State: Initialized, Role: FOLLOWER
I20260808 07:10:13.603770 32392 consensus_queue.cc:260] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } }
I20260808 07:10:13.603993 32397 tablet_bootstrap.cc:654] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:13.604081 32392 ts_tablet_manager.cc:1428] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:13.604133 32173 heartbeater.cc:503] Master 127.30.54.126:32845 was elected leader, sending a full tablet report...
I20260808 07:10:13.604854 32397 log.cc:824] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:13.605545 32397 tablet_bootstrap.cc:492] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421: No bootstrap required, opened a new log
I20260808 07:10:13.605495 32389 raft_consensus.cc:348] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } }
I20260808 07:10:13.605609 32389 raft_consensus.cc:374] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:13.605626 32397 ts_tablet_manager.cc:1397] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:13.605669 32389 raft_consensus.cc:729] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332c94d2831e48ff8b45ff45a62e2921, State: Initialized, Role: FOLLOWER
I20260808 07:10:13.605798 32389 consensus_queue.cc:260] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } }
I20260808 07:10:13.606088 32389 ts_tablet_manager.cc:1428] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:13.606130 31989 heartbeater.cc:503] Master 127.30.54.126:32845 was elected leader, sending a full tablet report...
I20260808 07:10:13.607519 32397 raft_consensus.cc:348] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } }
I20260808 07:10:13.607641 32397 raft_consensus.cc:374] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:13.607663 32397 raft_consensus.cc:729] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3cfbbe6082349aa8288bf69eb44f421, State: Initialized, Role: FOLLOWER
I20260808 07:10:13.607764 32397 consensus_queue.cc:260] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } }
I20260808 07:10:13.607988 32397 ts_tablet_manager.cc:1428] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 07:10:13.608042 32366 heartbeater.cc:503] Master 127.30.54.126:32845 was elected leader, sending a full tablet report...
I20260808 07:10:13.625355 32400 raft_consensus.cc:482] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:13.625437 32400 raft_consensus.cc:504] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } }
I20260808 07:10:13.625764 32400 leader_election.cc:288] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 473db6eb65cf417d941fb3f58c758516 (127.30.54.66:37887), b3cfbbe6082349aa8288bf69eb44f421 (127.30.54.67:38405)
I20260808 07:10:13.629652 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.626023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:54913 (local address 127.30.54.66:37887)
0808 07:10:13.626073 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.626075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.626088 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:10:13.626117 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.626118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.626130 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.626186 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:13.626407 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.627192 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.628715 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.629277 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.629358 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.629368 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.629548 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.629589 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:10:13.629652 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.626195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:37471 (local address 127.30.54.67:38405)
0808 07:10:13.626234 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.626236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.626247 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:13.626316 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.626317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.626325 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.626410 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:13.626542 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.627252 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.628714 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.629310 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.629401 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.629409 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.629553 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.629589 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:10:13.629739 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.626051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:38405 (local address 127.30.54.65:37471)
0808 07:10:13.626205 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.626215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:13.626259 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:13.626420 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:13.626421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:13.626422 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:13.626477 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.626478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.627330 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.627331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.628614 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.628617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.629396 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.629396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.629483 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.629487 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:13.629538 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:10:13.629567 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":45,"thread_start_us":96,"threads_started":1}
I20260808 07:10:13.629683 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.625945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:37887 (local address 127.30.54.65:54913)
0808 07:10:13.625996 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.626009 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:13.626063 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:13.626243 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:13.626244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:13.626246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:13.626317 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.626320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.627263 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.627264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.628614 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:13.628617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.629356 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.629368 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:13.629454 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.629459 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:13.629523 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:10:13.629567 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":63}
I20260808 07:10:13.630079 32294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912e535a382447dc92b720f4a0d87cb9" candidate_uuid: "332c94d2831e48ff8b45ff45a62e2921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3cfbbe6082349aa8288bf69eb44f421" is_pre_election: true
I20260808 07:10:13.630223 32294 raft_consensus.cc:2454] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 332c94d2831e48ff8b45ff45a62e2921 in term 0.
I20260808 07:10:13.630234 32107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912e535a382447dc92b720f4a0d87cb9" candidate_uuid: "332c94d2831e48ff8b45ff45a62e2921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "473db6eb65cf417d941fb3f58c758516" is_pre_election: true
I20260808 07:10:13.630383 32107 raft_consensus.cc:2454] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 332c94d2831e48ff8b45ff45a62e2921 in term 0.
I20260808 07:10:13.630509 31832 leader_election.cc:302] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 332c94d2831e48ff8b45ff45a62e2921, b3cfbbe6082349aa8288bf69eb44f421; no voters:
I20260808 07:10:13.630628 32400 raft_consensus.cc:2781] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:13.630682 32400 raft_consensus.cc:482] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:13.630712 32400 raft_consensus.cc:3037] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:13.631381 32400 raft_consensus.cc:504] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } }
I20260808 07:10:13.631498 32400 leader_election.cc:288] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [CANDIDATE]: Term 1 election: Requested vote from peers 473db6eb65cf417d941fb3f58c758516 (127.30.54.66:37887), b3cfbbe6082349aa8288bf69eb44f421 (127.30.54.67:38405)
I20260808 07:10:13.631687 32107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912e535a382447dc92b720f4a0d87cb9" candidate_uuid: "332c94d2831e48ff8b45ff45a62e2921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "473db6eb65cf417d941fb3f58c758516"
I20260808 07:10:13.631700 32294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "912e535a382447dc92b720f4a0d87cb9" candidate_uuid: "332c94d2831e48ff8b45ff45a62e2921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3cfbbe6082349aa8288bf69eb44f421"
I20260808 07:10:13.631767 32294 raft_consensus.cc:3037] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:13.631770 32107 raft_consensus.cc:3037] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:13.632464 32294 raft_consensus.cc:2454] T 912e535a382447dc92b720f4a0d87cb9 P b3cfbbe6082349aa8288bf69eb44f421 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 332c94d2831e48ff8b45ff45a62e2921 in term 1.
I20260808 07:10:13.632583 32107 raft_consensus.cc:2454] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 332c94d2831e48ff8b45ff45a62e2921 in term 1.
I20260808 07:10:13.632615 31832 leader_election.cc:302] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 332c94d2831e48ff8b45ff45a62e2921, b3cfbbe6082349aa8288bf69eb44f421; no voters:
I20260808 07:10:13.632711 32400 raft_consensus.cc:2781] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:13.632879 32400 raft_consensus.cc:686] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [term 1 LEADER]: Becoming Leader. State: Replica: 332c94d2831e48ff8b45ff45a62e2921, State: Running, Role: LEADER
I20260808 07:10:13.632985 32400 consensus_queue.cc:237] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } }
I20260808 07:10:13.633760 31727 catalog_manager.cc:5495] T 912e535a382447dc92b720f4a0d87cb9 P 332c94d2831e48ff8b45ff45a62e2921 reported cstate change: term changed from 0 to 1, leader changed from <none> to 332c94d2831e48ff8b45ff45a62e2921 (127.30.54.65). New cstate: current_term: 1 leader_uuid: "332c94d2831e48ff8b45ff45a62e2921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332c94d2831e48ff8b45ff45a62e2921" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "473db6eb65cf417d941fb3f58c758516" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3cfbbe6082349aa8288bf69eb44f421" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38405 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:13.643620 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.636590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37546 (local address 127.30.54.65:35421)
0808 07:10:13.636673 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.636675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.636768 (+ 93us) server_negotiation.cc:407] Connection header received
0808 07:10:13.636842 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.636843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.636853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.636902 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:13.637457 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.637905 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.638604 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.638843 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.638903 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.641094 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:13.641102 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:13.641103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:13.641109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:13.642651 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.643004 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.643007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.643007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.643060 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.643206 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.643206 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.643207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.643330 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:13.643426 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.643501 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.643562 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 07:10:13.643852 32174 tablet.cc:2369] T 912e535a382447dc92b720f4a0d87cb9 P 473db6eb65cf417d941fb3f58c758516: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:13.666715 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.660097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37560 (local address 127.30.54.65:35421)
0808 07:10:13.660186 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.660188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.660253 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:10:13.660300 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.660301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.660309 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.660366 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:13.660906 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.661387 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.662127 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.662320 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.662375 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.664550 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:13.664551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:13.664552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:13.664558 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:13.665709 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.666015 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.666016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.666016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.666075 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.666232 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.666233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.666233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.666319 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:13.666390 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.666528 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.666590 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[31659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:13.688869 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.682669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.30.54.65:35421)
0808 07:10:13.682727 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:13.682729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:13.682804 (+ 75us) server_negotiation.cc:407] Connection header received
0808 07:10:13.682884 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:13.682884 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:13.682893 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:13.682952 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:13.683435 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.683881 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.684543 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:13.684715 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:13.684763 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:13.686686 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:13.686688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:13.686688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:13.686694 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:13.688050 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.688340 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.688340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.688341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.688388 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:13.688534 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:13.688534 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:13.688535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:13.688607 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:13.688674 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:13.688755 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:10:13.688808 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 07:10:13.689296 31868 tablet_service.cc:2908] Rejecting scan request for tablet 912e535a382447dc92b720f4a0d87cb9: Uninitialized: clean time has not yet been initialized
I20260808 07:10:13.690127 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31812
I20260808 07:10:13.705363 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31994
I20260808 07:10:13.714802 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 32178
I20260808 07:10:13.720492 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31681
2026-08-08T07:10:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2131 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[32431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[32431](info): set up 2 sockets
Aug 08 07:10:13 dist-test-slave-znh6 krb5kdc[32431](info): commencing operation
krb5kdc: starting...
W20260808 07:10:14.946208 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:15.069653 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:36565
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:36427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:36565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:15.152304 32449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:15.152513 32449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:15.152539 32449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:15.153712 32449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:15.153746 32449 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:10:15.153760 32449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:15.153774 32449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:15.153785 32449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:15.153836 32449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:15.155311 32449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:36427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:36565
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:36565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:15.155551 32449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:15.155773 32449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:15.160792 32463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:15.160756 32449 server_base.cc:1034] running on GCE node
W20260808 07:10:15.160753 32457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:15.160763 32456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:15.161296 32449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:15.161619 32449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:15.162771 32449 hybrid_clock.cc:648] HybridClock initialized: now 1786173015162761 us; error 31 us; skew 500 ppm
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:15.168582 32449 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:15.169044 32449 webserver.cc:460] Webserver started at http://127.30.54.126:46773/ using document root <none> and password file <none>
I20260808 07:10:15.169281 32449 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:15.169368 32449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:15.169518 32449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:15.170529 32449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "c8dd7193124741d5bae066d33f741609"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "66da339f8e0008d25b289eda8da877fb"
server_key_iv: "7e7cb443c4ab5622ba492fe58c93e76a"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.170943 32449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "c8dd7193124741d5bae066d33f741609"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "66da339f8e0008d25b289eda8da877fb"
server_key_iv: "7e7cb443c4ab5622ba492fe58c93e76a"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.172294 32449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:15.173120 32473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:15.173301 32449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:15.173417 32449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "c8dd7193124741d5bae066d33f741609"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "66da339f8e0008d25b289eda8da877fb"
server_key_iv: "7e7cb443c4ab5622ba492fe58c93e76a"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.173498 32449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:15.195652 32449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:15.197454 32449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:15.197610 32449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:15.202355 32449 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:36565
I20260808 07:10:15.202423 32563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:36565 every 8 connection(s)
I20260808 07:10:15.202728 32449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:15.203398 32565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:15.204515 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 32449
I20260808 07:10:15.204614 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:15.204789 30937 external_mini_cluster.cc:1428] Setting key 4cf019b5a42a22f87102b4f0a7825dd1
I20260808 07:10:15.206099 32565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609: Bootstrap starting.
I20260808 07:10:15.206681 32565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:15.207029 32565 log.cc:824] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:15.207718 32565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609: No bootstrap required, opened a new log
I20260808 07:10:15.209527 32565 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8dd7193124741d5bae066d33f741609" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36565 } }
I20260808 07:10:15.209729 32565 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:15.209805 32565 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8dd7193124741d5bae066d33f741609, State: Initialized, Role: FOLLOWER
I20260808 07:10:15.209957 32565 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8dd7193124741d5bae066d33f741609" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36565 } }
I20260808 07:10:15.210043 32565 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:15.210088 32565 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:15.210134 32565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:15.210630 32565 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8dd7193124741d5bae066d33f741609" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36565 } }
I20260808 07:10:15.210750 32565 leader_election.cc:302] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8dd7193124741d5bae066d33f741609; no voters:
I20260808 07:10:15.210975 32565 leader_election.cc:288] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:15.211035 32570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:15.211205 32570 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [term 1 LEADER]: Becoming Leader. State: Replica: c8dd7193124741d5bae066d33f741609, State: Running, Role: LEADER
I20260808 07:10:15.211233 32565 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:15.211292 32570 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8dd7193124741d5bae066d33f741609" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36565 } }
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:15.211706 32571 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8dd7193124741d5bae066d33f741609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8dd7193124741d5bae066d33f741609" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36565 } } }
I20260808 07:10:15.211805 32571 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:15.211702 32573 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8dd7193124741d5bae066d33f741609. Latest consensus state: current_term: 1 leader_uuid: "c8dd7193124741d5bae066d33f741609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8dd7193124741d5bae066d33f741609" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36565 } } }
I20260808 07:10:15.212041 32573 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:15.212155 32581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:15.212891 32581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:15.214525 32581 catalog_manager.cc:1347] Generated new cluster ID: 7e799236640943209569307302db4a33
I20260808 07:10:15.214584 32581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:15.216122 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.205288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.30.54.126:36565)
0808 07:10:15.205504 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.205512 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.205536 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:10:15.205953 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.205959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.206001 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.206141 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:15.206774 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.207947 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.208672 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.209087 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.209209 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.211994 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:15.211998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:15.211999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:15.212011 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:15.214414 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.214852 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.214855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.214855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.214988 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.215201 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.215202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.215202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.215499 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:15.215652 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.215766 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.215915 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":99,"threads_started":1}
I20260808 07:10:15.218225 32581 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:15.218689 32581 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:15.224879 32581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609: Generated new TSK 0
I20260808 07:10:15.225044 32581 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:10:15.253665 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36565
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:36427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:15.334579 32597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:15.334767 32597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:15.334791 32597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:15.335929 32597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:15.335970 32597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:15.335991 32597 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:10:15.336009 32597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:15.337359 32597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:36427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.54.126:36565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:15.337532 32597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:15.337702 32597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:15.342754 32605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:15.342769 32603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:15.342773 32602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:15.342875 32597 server_base.cc:1034] running on GCE node
I20260808 07:10:15.343223 32597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:15.343501 32597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:15.344667 32597 hybrid_clock.cc:648] HybridClock initialized: now 1786173015344663 us; error 44 us; skew 500 ppm
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:15.347224 32597 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:10:15.347697 32597 webserver.cc:460] Webserver started at http://127.30.54.65:41881/ using document root <none> and password file <none>
I20260808 07:10:15.347990 32597 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:15.348057 32597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:15.348193 32597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:15.349206 32597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "031b591d722d4083a83098d237a43db5"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "2e3133748bebce35ce3cfa60f000d783"
server_key_iv: "ab9837e71864c10f96c7ef6cb228d7f7"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.349572 32597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "031b591d722d4083a83098d237a43db5"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "2e3133748bebce35ce3cfa60f000d783"
server_key_iv: "ab9837e71864c10f96c7ef6cb228d7f7"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.350944 32597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:10:15.351660 32616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:15.351846 32597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:15.351943 32597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "031b591d722d4083a83098d237a43db5"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "2e3133748bebce35ce3cfa60f000d783"
server_key_iv: "ab9837e71864c10f96c7ef6cb228d7f7"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.352025 32597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:15.371423 32597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:15.373214 32597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:15.373380 32597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:15.373939 32597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:15.373997 32597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:15.374039 32597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:15.374061 32597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:15.381662 32597 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:38619
I20260808 07:10:15.381753 303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:38619 every 8 connection(s)
I20260808 07:10:15.382059 32597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:15.384006 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.374318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:34163 (local address 127.30.54.126:36565)
0808 07:10:15.374387 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.374390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.374628 (+ 238us) server_negotiation.cc:407] Connection header received
0808 07:10:15.375144 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.375146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.375161 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.375247 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:15.375858 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.376594 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.377677 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.378030 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.378104 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.381425 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:15.381427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:15.381427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:15.381435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:15.382944 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.383311 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.383312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.383312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.383388 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.383559 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.383559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.383559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.383653 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:15.383698 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.383875 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.383942 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:10:15.384166 32627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.374293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36565 (local address 127.30.54.65:34163)
0808 07:10:15.374490 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.374530 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:15.375048 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:15.375298 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:15.375301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:15.375586 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:15.375762 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.375766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.376682 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.376683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.377592 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.377595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.378129 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.378130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.378339 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.378915 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:15.378925 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:15.381303 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:15.383023 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.383028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.383030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.383234 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.383433 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.383433 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.383434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.383521 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.383688 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:15.383691 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:15.383788 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 07:10:15.383863 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:10:15.383975 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":132,"threads_started":1}
I20260808 07:10:15.384670 304 heartbeater.cc:347] Connected to a master server at 127.30.54.126:36565
I20260808 07:10:15.384768 304 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:15.384946 304 heartbeater.cc:511] Master 127.30.54.126:36565 requested a full tablet report, sending...
I20260808 07:10:15.385694 32495 ts_manager.cc:194] Registered new tserver with Master: 031b591d722d4083a83098d237a43db5 (127.30.54.65:38619)
I20260808 07:10:15.386565 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:34163
I20260808 07:10:15.388492 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 32597
I20260808 07:10:15.388602 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:15.388764 30937 external_mini_cluster.cc:1428] Setting key 041b195ea1c1e41fe416d04ada2afda9
I20260808 07:10:15.391706 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.385724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:38267 (local address 127.30.54.126:36565)
0808 07:10:15.385789 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.385791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.385954 (+ 163us) server_negotiation.cc:407] Connection header received
0808 07:10:15.385991 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.385992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.386001 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.386044 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:15.386375 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.386985 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.387809 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.387982 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.388029 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.389012 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:15.389014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:15.389014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:15.389021 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:15.390600 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.390964 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.390965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.390965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.391028 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.391235 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.391236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.391237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.391378 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:15.391444 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.391597 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.391653 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:10:15.391827 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.385680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36565 (local address 127.30.54.65:38267)
0808 07:10:15.385875 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.385887 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:15.385951 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:15.386058 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:15.386058 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:15.386259 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:15.386316 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.386318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.387066 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.387067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.387742 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.387744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.388040 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.388041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.388098 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.388537 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:15.388542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:15.388914 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:15.390676 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.390677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.390677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.390891 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.391094 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.391094 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.391095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.391166 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.391447 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:15.391449 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:15.391498 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:10:15.391580 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:10:15.391693 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":28,"thread_start_us":138,"threads_started":1}
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:10:15.415884 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36565
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:36427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:15.496245 318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:15.496445 318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:15.496470 318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:15.497591 318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:15.497633 318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:15.497653 318 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:10:15.497671 318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:15.499087 318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:36427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.54.126:36565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:15.499326 318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:15.499540 318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:10:15.504973 318 server_base.cc:1034] running on GCE node
W20260808 07:10:15.504889 325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:15.504927 328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:15.504892 324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:15.505345 318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:15.505570 318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:15.506712 318 hybrid_clock.cc:648] HybridClock initialized: now 1786173015506701 us; error 28 us; skew 500 ppm
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:15.509424 318 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:10:15.509852 318 webserver.cc:460] Webserver started at http://127.30.54.66:45671/ using document root <none> and password file <none>
I20260808 07:10:15.510066 318 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:15.510110 318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:15.510251 318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:15.511320 318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "67d2cef3d2e54520b0a4da67cf2be46f"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "d03c22695ef668576834402b633ee845"
server_key_iv: "38b2b8c3ac7dfc2de30f56a74e183a9b"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.511696 318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "67d2cef3d2e54520b0a4da67cf2be46f"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "d03c22695ef668576834402b633ee845"
server_key_iv: "38b2b8c3ac7dfc2de30f56a74e183a9b"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.513063 318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:15.513772 337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:15.513942 318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:15.514035 318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "67d2cef3d2e54520b0a4da67cf2be46f"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "d03c22695ef668576834402b633ee845"
server_key_iv: "38b2b8c3ac7dfc2de30f56a74e183a9b"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.514109 318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:15.527470 318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:15.529558 318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:15.529707 318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:15.530220 318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:15.530274 318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:15.530311 318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:15.530335 318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:15.537472 318 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:46267
I20260808 07:10:15.537555 503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:46267 every 8 connection(s)
I20260808 07:10:15.537829 318 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:15.539965 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.530586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:41459 (local address 127.30.54.126:36565)
0808 07:10:15.530647 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.530650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.530901 (+ 251us) server_negotiation.cc:407] Connection header received
0808 07:10:15.531335 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.531337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.531352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.531429 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:15.531970 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.532767 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.534441 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.534663 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.534754 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.537419 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:15.537420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:15.537420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:15.537429 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:15.538921 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.539280 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.539280 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.539281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.539332 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.539514 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.539514 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.539514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.539611 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:15.539661 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.539838 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.539895 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:10:15.540110 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.530568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36565 (local address 127.30.54.66:41459)
0808 07:10:15.530730 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.530768 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:15.531243 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:15.531496 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:15.531499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:15.531711 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:15.531875 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.531878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.532889 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.532890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.534351 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.534355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.534777 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.534778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.534937 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.535398 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:15.535422 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:15.537316 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:15.538989 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.538993 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.538995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.539224 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.539375 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.539376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.539376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.539463 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.539651 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:15.539653 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:15.539746 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 07:10:15.539824 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:10:15.539937 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260808 07:10:15.540580 504 heartbeater.cc:347] Connected to a master server at 127.30.54.126:36565
I20260808 07:10:15.540663 504 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:15.540668 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 318
I20260808 07:10:15.540764 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:15.540843 504 heartbeater.cc:511] Master 127.30.54.126:36565 requested a full tablet report, sending...
I20260808 07:10:15.540951 30937 external_mini_cluster.cc:1428] Setting key fa16084374dc427d421e6a014914c26f
I20260808 07:10:15.541241 32494 ts_manager.cc:194] Registered new tserver with Master: 67d2cef3d2e54520b0a4da67cf2be46f (127.30.54.66:46267)
I20260808 07:10:15.541661 32494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:41459
I20260808 07:10:15.547552 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.541926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:59919 (local address 127.30.54.126:36565)
0808 07:10:15.541978 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.541980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.542162 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:10:15.542223 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.542223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.542233 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.542288 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:15.542590 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.543108 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.543849 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.544037 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.544104 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.544963 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:15.544964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:15.544965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:15.544972 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:15.546433 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.546762 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.546763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.546764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.546845 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.547023 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.547024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.547025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.547166 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:15.547240 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.547388 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.547463 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:10:15.547560 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.541890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36565 (local address 127.30.54.66:59919)
0808 07:10:15.542088 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.542098 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:15.542174 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:15.542293 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:15.542294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:15.542469 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:15.542518 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.542519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.543194 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.543196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.543766 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.543768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.544104 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.544105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.544177 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.544561 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:15.544566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:15.544883 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:15.546511 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.546512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.546512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.546680 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.546896 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.546897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.546897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.546954 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.547233 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:15.547234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:15.547278 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:10:15.547361 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:10:15.547464 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:10:15.567514 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36565
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:36427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:15.648205 517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:15.648398 517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:15.648425 517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:15.649503 517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:15.649536 517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:15.649556 517 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:10:15.649573 517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:15.650933 517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:36427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.54.126:36565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:15.651099 517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:15.651261 517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:15.656533 523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:15.656543 526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:15.656571 524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:15.656862 517 server_base.cc:1034] running on GCE node
I20260808 07:10:15.657028 517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:15.657245 517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:15.658413 517 hybrid_clock.cc:648] HybridClock initialized: now 1786173015658397 us; error 28 us; skew 500 ppm
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:15.660919 517 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:10:15.661386 517 webserver.cc:460] Webserver started at http://127.30.54.67:39577/ using document root <none> and password file <none>
I20260808 07:10:15.661635 517 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:15.661701 517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:15.661850 517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:15.663054 517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "bf133e6848f64edb89f28494e17d2ff5"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "a55082eef5d74604c2cf524ecb6d6096"
server_key_iv: "97cf792d9df52c43fb9b7e1d4b5a8301"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.663470 517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "bf133e6848f64edb89f28494e17d2ff5"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "a55082eef5d74604c2cf524ecb6d6096"
server_key_iv: "97cf792d9df52c43fb9b7e1d4b5a8301"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.664872 517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:10:15.665630 533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:15.665817 517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:15.665927 517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "bf133e6848f64edb89f28494e17d2ff5"
format_stamp: "Formatted at 2026-08-08 07:10:15 on dist-test-slave-znh6"
server_key: "a55082eef5d74604c2cf524ecb6d6096"
server_key_iv: "97cf792d9df52c43fb9b7e1d4b5a8301"
server_key_version: "encryptionkey@0"
I20260808 07:10:15.665997 517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:15.673483 517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:15.675330 517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:15.675478 517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:15.676039 517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:15.676092 517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:15.676131 517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:15.676154 517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:15.683640 517 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:36109
I20260808 07:10:15.684042 517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:15.684190 684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:36109 every 8 connection(s)
I20260808 07:10:15.686107 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.676479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:52341 (local address 127.30.54.126:36565)
0808 07:10:15.676534 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.676536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.676763 (+ 227us) server_negotiation.cc:407] Connection header received
0808 07:10:15.677468 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.677470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.677496 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.677563 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:15.678449 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.679017 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.680015 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.680229 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.680313 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.683325 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:15.683327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:15.683327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:15.683333 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:15.684894 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.685292 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.685293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.685294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.685346 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.685548 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.685548 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.685549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.685654 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:15.685704 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.685982 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.686043 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:10:15.686282 545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.676436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36565 (local address 127.30.54.67:52341)
0808 07:10:15.676609 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.676660 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:15.677353 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:15.677646 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:15.677651 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:15.678034 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:15.678332 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.678339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.679105 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.679106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.679924 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.679927 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.680323 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.680325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.680443 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.680924 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:15.680932 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:15.683226 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:15.684961 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.684966 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.684967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.685228 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.685416 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.685417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.685417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.685500 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.685709 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:15.685713 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:15.685877 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 07:10:15.685956 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:10:15.686070 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260808 07:10:15.686750 687 heartbeater.cc:347] Connected to a master server at 127.30.54.126:36565
I20260808 07:10:15.686857 687 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:15.687037 687 heartbeater.cc:511] Master 127.30.54.126:36565 requested a full tablet report, sending...
I20260808 07:10:15.687484 32495 ts_manager.cc:194] Registered new tserver with Master: bf133e6848f64edb89f28494e17d2ff5 (127.30.54.67:36109)
I20260808 07:10:15.688017 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:52341
I20260808 07:10:15.691993 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 517
I20260808 07:10:15.692097 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:15.692270 30937 external_mini_cluster.cc:1428] Setting key 8f7aa8c4dffd6c2ee8e57864e1474abc
I20260808 07:10:15.693037 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:15.693161 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.687851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:45187 (local address 127.30.54.126:36565)
0808 07:10:15.687892 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.687894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.687988 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:10:15.688054 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.688055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.688062 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.688127 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:15.688442 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.688898 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.689588 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.689778 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.689847 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.690690 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:15.690691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:15.690691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:15.690698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:15.692033 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.692411 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.692412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.692412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.692474 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.692674 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.692675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.692676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.692781 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:15.692853 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.693007 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.693075 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:15.693171 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.687789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36565 (local address 127.30.54.67:45187)
0808 07:10:15.687934 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.687944 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:15.687995 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:15.688154 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:15.688154 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:15.688316 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:15.688369 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.688371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.688986 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.688987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.689509 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.689511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.689847 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.689848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.689925 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.690310 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:15.690315 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:15.690587 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:15.692132 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.692133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.692133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.692309 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.692542 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:15.692543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:15.692543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:15.692605 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:15.692845 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:15.692846 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:15.692894 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 07:10:15.692978 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:10:15.693077 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:10:15 dist-test-slave-znh6 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:15.716733 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.709788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47984 (local address 127.30.54.126:36565)
0808 07:10:15.709842 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.709844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.709857 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:10:15.709911 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.709912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.709920 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.709971 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:15.710497 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.711078 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.711905 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.712136 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.712200 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.714291 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:15.714293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:15.714293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:15.714299 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:15.715672 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.716014 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.716015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.716015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.716063 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:15.716234 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:15.716235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:15.716235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:15.716344 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:15.716415 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.716532 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.716604 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:10:15.718218 32495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:15.719892 32495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:15.726279 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.722028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.30.54.67:36109)
0808 07:10:15.722227 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.722231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.722463 (+ 232us) server_negotiation.cc:407] Connection header received
0808 07:10:15.722511 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.722513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.722529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.722612 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:15.722803 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.723980 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.725106 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.725829 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.725916 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.725980 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.726145 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.726187 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
I20260808 07:10:15.726264 710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.721951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:38619 (local address 127.0.0.1:49434)
0808 07:10:15.722304 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.722313 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:15.722372 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:15.722945 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:15.722946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:15.722948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:15.722978 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.722979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.723839 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.723840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.725098 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.725101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.725974 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.725975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.726042 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.726046 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:15.726079 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 07:10:15.726148 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":6,"spinlock_wait_cycles":384,"thread_start_us":62,"threads_started":1}
I20260808 07:10:15.726274 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.722360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.30.54.66:46267)
0808 07:10:15.722558 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.722562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.722571 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:10:15.722609 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.722610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.722625 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.722699 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:15.722940 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.724029 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.725342 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.725824 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.725911 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.725969 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.726125 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.726168 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260808 07:10:15.726269 712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.722154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46267 (local address 127.0.0.1:58878)
0808 07:10:15.722390 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.722403 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:15.722459 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:15.722769 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:15.722771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:15.722773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:15.722868 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.722870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.724106 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.724107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.725253 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.725255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.725919 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.725920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.726001 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.726015 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:10:15.726106 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:10:15.726149 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":25,"thread_start_us":51,"threads_started":1}
I20260808 07:10:15.726480 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.721951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:36109 (local address 127.0.0.1:48198)
0808 07:10:15.722258 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.722301 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:15.722372 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:15.722633 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:15.722636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:15.722639 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:15.722706 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.722708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.724088 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.724089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.725027 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:15.725029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.726008 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.726009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:15.726057 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.726060 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:15.726355 (+ 295us) client_negotiation.cc:240] Negotiation successful
0808 07:10:15.726402 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":37,"thread_start_us":54,"threads_started":1}
I20260808 07:10:15.726681 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.722148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49434 (local address 127.30.54.65:38619)
0808 07:10:15.722743 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:15.722747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:15.722757 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:15.722793 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:15.722794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:15.722832 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:15.723045 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:15.723092 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.723753 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.725180 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:15.725878 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:15.726479 (+ 601us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:15.726548 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:15.726578 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 07:10:15.726613 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":79,"threads_started":1}
I20260808 07:10:15.727283 599 tablet_service.cc:1467] Processing CreateTablet for tablet 5a65dc385c154a74818b2ae79f642a75 (DEFAULT_TABLE table=test-table [id=8608b075eb8048e19ef17fcfd87fdcc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:15.727326 32679 tablet_service.cc:1467] Processing CreateTablet for tablet 5a65dc385c154a74818b2ae79f642a75 (DEFAULT_TABLE table=test-table [id=8608b075eb8048e19ef17fcfd87fdcc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:15.727658 32679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a65dc385c154a74818b2ae79f642a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:15.727662 599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a65dc385c154a74818b2ae79f642a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:15.727718 409 tablet_service.cc:1467] Processing CreateTablet for tablet 5a65dc385c154a74818b2ae79f642a75 (DEFAULT_TABLE table=test-table [id=8608b075eb8048e19ef17fcfd87fdcc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:15.728018 409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a65dc385c154a74818b2ae79f642a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:15.730353 717 tablet_bootstrap.cc:492] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: Bootstrap starting.
I20260808 07:10:15.730902 718 tablet_bootstrap.cc:492] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: Bootstrap starting.
I20260808 07:10:15.731174 717 tablet_bootstrap.cc:654] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:15.731583 717 log.cc:824] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:15.731583 720 tablet_bootstrap.cc:492] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: Bootstrap starting.
I20260808 07:10:15.731796 718 tablet_bootstrap.cc:654] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:15.732115 718 log.cc:824] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:15.732362 720 tablet_bootstrap.cc:654] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:15.732445 717 tablet_bootstrap.cc:492] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: No bootstrap required, opened a new log
I20260808 07:10:15.732517 717 ts_tablet_manager.cc:1397] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:15.732688 720 log.cc:824] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:15.732868 718 tablet_bootstrap.cc:492] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: No bootstrap required, opened a new log
I20260808 07:10:15.732944 718 ts_tablet_manager.cc:1397] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:10:15.733505 720 tablet_bootstrap.cc:492] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: No bootstrap required, opened a new log
I20260808 07:10:15.733577 720 ts_tablet_manager.cc:1397] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:15.734284 717 raft_consensus.cc:348] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } }
I20260808 07:10:15.734439 717 raft_consensus.cc:374] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:15.734485 717 raft_consensus.cc:729] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031b591d722d4083a83098d237a43db5, State: Initialized, Role: FOLLOWER
I20260808 07:10:15.734597 717 consensus_queue.cc:260] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } }
I20260808 07:10:15.734903 717 ts_tablet_manager.cc:1428] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:10:15.734913 718 raft_consensus.cc:348] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } }
I20260808 07:10:15.735042 304 heartbeater.cc:503] Master 127.30.54.126:36565 was elected leader, sending a full tablet report...
I20260808 07:10:15.735073 718 raft_consensus.cc:374] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:15.735116 718 raft_consensus.cc:729] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf133e6848f64edb89f28494e17d2ff5, State: Initialized, Role: FOLLOWER
I20260808 07:10:15.735241 718 consensus_queue.cc:260] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } }
I20260808 07:10:15.735513 718 ts_tablet_manager.cc:1428] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:10:15.735538 687 heartbeater.cc:503] Master 127.30.54.126:36565 was elected leader, sending a full tablet report...
I20260808 07:10:15.735548 720 raft_consensus.cc:348] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } }
I20260808 07:10:15.735682 720 raft_consensus.cc:374] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:15.735704 720 raft_consensus.cc:729] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d2cef3d2e54520b0a4da67cf2be46f, State: Initialized, Role: FOLLOWER
I20260808 07:10:15.735833 720 consensus_queue.cc:260] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } }
I20260808 07:10:15.736148 504 heartbeater.cc:503] Master 127.30.54.126:36565 was elected leader, sending a full tablet report...
I20260808 07:10:15.736132 720 ts_tablet_manager.cc:1428] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260808 07:10:15.788414 506 tablet.cc:2369] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:15.883437 306 tablet.cc:2369] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:15.934665 690 tablet.cc:2369] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:16.010476 727 raft_consensus.cc:482] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:16.010605 727 raft_consensus.cc:504] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } }
I20260808 07:10:16.011037 727 leader_election.cc:288] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf133e6848f64edb89f28494e17d2ff5 (127.30.54.67:36109), 031b591d722d4083a83098d237a43db5 (127.30.54.65:38619)
I20260808 07:10:16.014585 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.011301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:41283 (local address 127.30.54.65:38619)
0808 07:10:16.011375 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.011377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.011575 (+ 198us) server_negotiation.cc:407] Connection header received
0808 07:10:16.011612 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.011613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.011624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.011687 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:16.011872 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.012493 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.013831 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.014205 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.014293 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.014302 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.014464 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.014503 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:10:16.014593 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.011342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:42835 (local address 127.30.54.67:36109)
0808 07:10:16.011388 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.011390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.011401 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:16.011476 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.011477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.011487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.011559 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:16.011775 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.012286 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.013806 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.014205 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.014293 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.014302 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.014482 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.014521 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:10:16.014599 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.011243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:36109 (local address 127.30.54.66:42835)
0808 07:10:16.011315 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.011328 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.011403 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.011606 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.011607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.011608 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:16.011692 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.011695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.012415 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.012416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.013702 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.013705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.014289 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.014290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.014404 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.014407 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.014475 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.014518 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 07:10:16.014626 729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.011247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:38619 (local address 127.30.54.66:41283)
0808 07:10:16.011499 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.011509 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.011567 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.011697 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.011698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.011699 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:16.011788 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.011790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.012571 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.012572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.013726 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.013728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.014294 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.014295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.014386 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.014389 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.014458 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.014501 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":23,"thread_start_us":106,"threads_started":1}
I20260808 07:10:16.015034 32703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a65dc385c154a74818b2ae79f642a75" candidate_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031b591d722d4083a83098d237a43db5" is_pre_election: true
I20260808 07:10:16.015048 628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a65dc385c154a74818b2ae79f642a75" candidate_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf133e6848f64edb89f28494e17d2ff5" is_pre_election: true
I20260808 07:10:16.015205 32703 raft_consensus.cc:2454] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d2cef3d2e54520b0a4da67cf2be46f in term 0.
I20260808 07:10:16.015211 628 raft_consensus.cc:2454] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67d2cef3d2e54520b0a4da67cf2be46f in term 0.
I20260808 07:10:16.015455 347 leader_election.cc:302] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031b591d722d4083a83098d237a43db5, 67d2cef3d2e54520b0a4da67cf2be46f; no voters:
I20260808 07:10:16.015651 727 raft_consensus.cc:2781] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:16.015692 727 raft_consensus.cc:482] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:16.015708 727 raft_consensus.cc:3037] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:16.016609 727 raft_consensus.cc:504] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } }
I20260808 07:10:16.016803 727 leader_election.cc:288] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [CANDIDATE]: Term 1 election: Requested vote from peers bf133e6848f64edb89f28494e17d2ff5 (127.30.54.67:36109), 031b591d722d4083a83098d237a43db5 (127.30.54.65:38619)
I20260808 07:10:16.016947 628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a65dc385c154a74818b2ae79f642a75" candidate_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf133e6848f64edb89f28494e17d2ff5"
I20260808 07:10:16.016969 32703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a65dc385c154a74818b2ae79f642a75" candidate_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031b591d722d4083a83098d237a43db5"
I20260808 07:10:16.017032 628 raft_consensus.cc:3037] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:16.017060 32703 raft_consensus.cc:3037] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:16.017741 628 raft_consensus.cc:2454] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d2cef3d2e54520b0a4da67cf2be46f in term 1.
I20260808 07:10:16.017901 32703 raft_consensus.cc:2454] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67d2cef3d2e54520b0a4da67cf2be46f in term 1.
I20260808 07:10:16.017922 341 leader_election.cc:302] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67d2cef3d2e54520b0a4da67cf2be46f, bf133e6848f64edb89f28494e17d2ff5; no voters:
I20260808 07:10:16.018013 727 raft_consensus.cc:2781] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:16.018168 727 raft_consensus.cc:686] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 LEADER]: Becoming Leader. State: Replica: 67d2cef3d2e54520b0a4da67cf2be46f, State: Running, Role: LEADER
I20260808 07:10:16.018267 727 consensus_queue.cc:237] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } }
I20260808 07:10:16.019120 32495 catalog_manager.cc:5495] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f reported cstate change: term changed from 0 to 1, leader changed from <none> to 67d2cef3d2e54520b0a4da67cf2be46f (127.30.54.66). New cstate: current_term: 1 leader_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:16.080745 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.078374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.30.54.66:46267)
0808 07:10:16.078448 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.078450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.078460 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:16.078495 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.078496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.078511 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.078576 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:16.078755 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.079247 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.080024 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.080207 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.080258 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.080458 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:16.080544 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:16.080600 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.080667 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.080686 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:10:16.082544 628 raft_consensus.cc:1261] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Refusing update from remote peer 67d2cef3d2e54520b0a4da67cf2be46f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:16.082545 32703 raft_consensus.cc:1261] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [term 1 FOLLOWER]: Refusing update from remote peer 67d2cef3d2e54520b0a4da67cf2be46f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:16.082939 727 consensus_queue.cc:1035] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:16.083086 731 consensus_queue.cc:1035] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [LEADER]: Connected to new peer: Peer: permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:16.085497 736 mvcc.cc:205] Tried to move back new op lower bound from 7316164673871155200 to 7316164673611505664. Current Snapshot: MvccSnapshot[applied={T|T < 7316164673871155200}]
I20260808 07:10:16.086352 739 mvcc.cc:205] Tried to move back new op lower bound from 7316164673871155200 to 7316164673611505664. Current Snapshot: MvccSnapshot[applied={T|T < 7316164673871155200 or (T in {7316164673871155200})}]
I20260808 07:10:16.089244 32494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:16.089326 32494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:16.090080 32494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609: Sending DeleteTablet for 3 replicas of tablet 5a65dc385c154a74818b2ae79f642a75
I20260808 07:10:16.090423 32679 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a65dc385c154a74818b2ae79f642a75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:16 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:49434
I20260808 07:10:16.090620 747 tablet_replica.cc:330] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: stopping tablet replica
I20260808 07:10:16.090620 599 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a65dc385c154a74818b2ae79f642a75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:16 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48198
I20260808 07:10:16.090729 747 raft_consensus.cc:2229] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:16.090866 747 raft_consensus.cc:2258] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:16.090893 409 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a65dc385c154a74818b2ae79f642a75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:16 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:58878
I20260808 07:10:16.090890 748 tablet_replica.cc:330] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: stopping tablet replica
I20260808 07:10:16.091115 748 raft_consensus.cc:2229] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:16.091279 748 raft_consensus.cc:2258] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:16.091356 747 ts_tablet_manager.cc:1905] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:16.091321 32494 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:16.091466 32494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:16.091941 748 ts_tablet_manager.cc:1905] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:16.092952 599 tablet_service.cc:1467] Processing CreateTablet for tablet ded50494d9fc4732a3ce1797c24da778 (DEFAULT_TABLE table=test-table [id=5a8cd1a2b84440e68ab9ca16fc0e0c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:16.093077 599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded50494d9fc4732a3ce1797c24da778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:16.093313 747 ts_tablet_manager.cc:1918] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:16.093387 747 log.cc:1196] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/5a65dc385c154a74818b2ae79f642a75
I20260808 07:10:16.093669 747 ts_tablet_manager.cc:1939] T 5a65dc385c154a74818b2ae79f642a75 P 031b591d722d4083a83098d237a43db5: Deleting consensus metadata
I20260808 07:10:16.094158 32479 catalog_manager.cc:4841] TS 031b591d722d4083a83098d237a43db5 (127.30.54.65:38619): tablet 5a65dc385c154a74818b2ae79f642a75 (table test-table [id=8608b075eb8048e19ef17fcfd87fdcc8]) successfully deleted
I20260808 07:10:16.094573 718 tablet_bootstrap.cc:492] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: Bootstrap starting.
I20260808 07:10:16.094583 748 ts_tablet_manager.cc:1918] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:16.094730 32679 tablet_service.cc:1467] Processing CreateTablet for tablet ded50494d9fc4732a3ce1797c24da778 (DEFAULT_TABLE table=test-table [id=5a8cd1a2b84440e68ab9ca16fc0e0c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:16.094862 748 log.cc:1196] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/5a65dc385c154a74818b2ae79f642a75
I20260808 07:10:16.094870 32679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded50494d9fc4732a3ce1797c24da778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:16.095031 718 tablet_bootstrap.cc:654] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:16.095366 748 ts_tablet_manager.cc:1939] T 5a65dc385c154a74818b2ae79f642a75 P bf133e6848f64edb89f28494e17d2ff5: Deleting consensus metadata
I20260808 07:10:16.095602 718 tablet_bootstrap.cc:492] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: No bootstrap required, opened a new log
I20260808 07:10:16.095660 718 ts_tablet_manager.cc:1397] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:16.095810 718 raft_consensus.cc:348] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.095880 718 raft_consensus.cc:374] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:16.095901 718 raft_consensus.cc:729] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf133e6848f64edb89f28494e17d2ff5, State: Initialized, Role: FOLLOWER
I20260808 07:10:16.095933 718 consensus_queue.cc:260] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.096027 718 ts_tablet_manager.cc:1428] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:16.096194 717 tablet_bootstrap.cc:492] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: Bootstrap starting.
I20260808 07:10:16.096509 717 tablet_bootstrap.cc:654] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:16.097007 32475 catalog_manager.cc:4841] TS bf133e6848f64edb89f28494e17d2ff5 (127.30.54.67:36109): tablet 5a65dc385c154a74818b2ae79f642a75 (table test-table [id=8608b075eb8048e19ef17fcfd87fdcc8]) successfully deleted
I20260808 07:10:16.097272 409 tablet_service.cc:1467] Processing CreateTablet for tablet ded50494d9fc4732a3ce1797c24da778 (DEFAULT_TABLE table=test-table [id=5a8cd1a2b84440e68ab9ca16fc0e0c6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:16.097390 409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded50494d9fc4732a3ce1797c24da778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:16.098289 720 tablet_bootstrap.cc:492] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: Bootstrap starting.
I20260808 07:10:16.098438 717 tablet_bootstrap.cc:492] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: No bootstrap required, opened a new log
I20260808 07:10:16.098497 717 ts_tablet_manager.cc:1397] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:16.098641 717 raft_consensus.cc:348] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.098729 717 raft_consensus.cc:374] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:16.098738 720 tablet_bootstrap.cc:654] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:16.098757 717 raft_consensus.cc:729] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031b591d722d4083a83098d237a43db5, State: Initialized, Role: FOLLOWER
I20260808 07:10:16.098803 717 consensus_queue.cc:260] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.099478 717 ts_tablet_manager.cc:1428] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:16.099668 720 tablet_bootstrap.cc:492] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: No bootstrap required, opened a new log
I20260808 07:10:16.099725 720 ts_tablet_manager.cc:1397] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:16.099810 752 tablet_replica.cc:330] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: stopping tablet replica
I20260808 07:10:16.099916 752 raft_consensus.cc:2229] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:16.099874 720 raft_consensus.cc:348] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.099964 720 raft_consensus.cc:374] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:16.099993 720 raft_consensus.cc:729] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d2cef3d2e54520b0a4da67cf2be46f, State: Initialized, Role: FOLLOWER
I20260808 07:10:16.100036 720 consensus_queue.cc:260] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.100132 720 ts_tablet_manager.cc:1428] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:16.100502 752 raft_consensus.cc:2258] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:16.100864 752 ts_tablet_manager.cc:1905] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:16.102450 752 ts_tablet_manager.cc:1918] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:16.102509 752 log.cc:1196] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/5a65dc385c154a74818b2ae79f642a75
I20260808 07:10:16.102684 752 ts_tablet_manager.cc:1939] T 5a65dc385c154a74818b2ae79f642a75 P 67d2cef3d2e54520b0a4da67cf2be46f: Deleting consensus metadata
I20260808 07:10:16.103096 32475 catalog_manager.cc:4841] TS 67d2cef3d2e54520b0a4da67cf2be46f (127.30.54.66:46267): tablet 5a65dc385c154a74818b2ae79f642a75 (table test-table [id=8608b075eb8048e19ef17fcfd87fdcc8]) successfully deleted
I20260808 07:10:16.153873 725 raft_consensus.cc:482] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:16.153990 725 raft_consensus.cc:504] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.154335 725 leader_election.cc:288] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67d2cef3d2e54520b0a4da67cf2be46f (127.30.54.66:46267), bf133e6848f64edb89f28494e17d2ff5 (127.30.54.67:36109)
I20260808 07:10:16.158020 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.154640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:46101 (local address 127.30.54.66:46267)
0808 07:10:16.154707 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.154709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.154880 (+ 171us) server_negotiation.cc:407] Connection header received
0808 07:10:16.155049 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.155050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.155062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.155132 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:16.155292 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.156087 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.157199 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.157627 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.157693 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.157702 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.157903 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.157940 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:10:16.158326 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.154583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46267 (local address 127.30.54.65:46101)
0808 07:10:16.154772 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.154783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.154912 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.155152 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.155152 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.155155 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:16.155234 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.155236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.156194 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.156195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.157108 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.157111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.157755 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.157756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.157825 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.157829 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.158173 (+ 344us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.158217 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":69,"thread_start_us":112,"threads_started":1}
I20260808 07:10:16.158428 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.154921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:45311 (local address 127.30.54.67:36109)
0808 07:10:16.154970 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.154972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.155070 (+ 98us) server_negotiation.cc:407] Connection header received
0808 07:10:16.155127 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.155128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.155139 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.155186 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:16.155366 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.156095 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.157525 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.158059 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.158129 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.158137 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.158314 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.158362 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:10:16.158447 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.154834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:36109 (local address 127.30.54.65:45311)
0808 07:10:16.155012 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.155022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.155073 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.155204 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.155205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.155208 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:16.155287 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.155289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.156187 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.156188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.157430 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.157433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.158173 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.158174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.158247 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.158250 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.158303 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.158339 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":18,"thread_start_us":90,"threads_started":1}
I20260808 07:10:16.158623 434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded50494d9fc4732a3ce1797c24da778" candidate_uuid: "031b591d722d4083a83098d237a43db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" is_pre_election: true
I20260808 07:10:16.158740 434 raft_consensus.cc:2454] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 031b591d722d4083a83098d237a43db5 in term 0.
I20260808 07:10:16.158841 628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded50494d9fc4732a3ce1797c24da778" candidate_uuid: "031b591d722d4083a83098d237a43db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf133e6848f64edb89f28494e17d2ff5" is_pre_election: true
I20260808 07:10:16.158931 628 raft_consensus.cc:2454] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 031b591d722d4083a83098d237a43db5 in term 0.
I20260808 07:10:16.158946 32619 leader_election.cc:302] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031b591d722d4083a83098d237a43db5, 67d2cef3d2e54520b0a4da67cf2be46f; no voters:
I20260808 07:10:16.159070 725 raft_consensus.cc:2781] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:16.159112 725 raft_consensus.cc:482] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:16.159148 725 raft_consensus.cc:3037] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:16.160084 725 raft_consensus.cc:504] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.160220 725 leader_election.cc:288] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [CANDIDATE]: Term 1 election: Requested vote from peers 67d2cef3d2e54520b0a4da67cf2be46f (127.30.54.66:46267), bf133e6848f64edb89f28494e17d2ff5 (127.30.54.67:36109)
I20260808 07:10:16.160403 434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded50494d9fc4732a3ce1797c24da778" candidate_uuid: "031b591d722d4083a83098d237a43db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d2cef3d2e54520b0a4da67cf2be46f"
I20260808 07:10:16.160446 628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded50494d9fc4732a3ce1797c24da778" candidate_uuid: "031b591d722d4083a83098d237a43db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf133e6848f64edb89f28494e17d2ff5"
I20260808 07:10:16.160498 434 raft_consensus.cc:3037] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:16.160523 628 raft_consensus.cc:3037] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:16.161123 434 raft_consensus.cc:2454] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 031b591d722d4083a83098d237a43db5 in term 1.
I20260808 07:10:16.161281 32619 leader_election.cc:302] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031b591d722d4083a83098d237a43db5, 67d2cef3d2e54520b0a4da67cf2be46f; no voters:
I20260808 07:10:16.161338 628 raft_consensus.cc:2454] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 031b591d722d4083a83098d237a43db5 in term 1.
I20260808 07:10:16.161422 725 raft_consensus.cc:2781] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:16.161579 725 raft_consensus.cc:686] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 1 LEADER]: Becoming Leader. State: Replica: 031b591d722d4083a83098d237a43db5, State: Running, Role: LEADER
I20260808 07:10:16.161655 725 consensus_queue.cc:237] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.162395 32495 catalog_manager.cc:5495] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 031b591d722d4083a83098d237a43db5 (127.30.54.65). New cstate: current_term: 1 leader_uuid: "031b591d722d4083a83098d237a43db5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:10:16 dist-test-slave-znh6 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:16.207603 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.199603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47994 (local address 127.30.54.126:36565)
0808 07:10:16.199659 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.199661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.199776 (+ 115us) server_negotiation.cc:407] Connection header received
0808 07:10:16.199859 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.199860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.199869 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.199924 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:16.200506 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.201051 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.202158 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.202447 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.202517 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.205053 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:16.205054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:16.205054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:16.205061 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:16.206397 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:16.206794 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:16.206795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:16.206796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:16.206873 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:16.207072 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:16.207072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:16.207073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:16.207180 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:16.207250 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.207396 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.207485 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:10:16.207681 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.199519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36565 (local address 127.0.0.1:47994)
0808 07:10:16.199694 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.199706 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.199796 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.199945 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.199946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.200360 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:16.200422 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.200425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.201141 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.201142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.202070 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.202073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.202532 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.202533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.202601 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.203223 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:16.203228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:16.204969 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:16.206511 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:16.206518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:16.206520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:16.206707 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:16.206949 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:16.206950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:16.206950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:16.207012 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:16.207240 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:16.207247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:16.207292 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.207373 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.207539 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":64,"thread_start_us":89,"threads_started":1}
I20260808 07:10:16.209769 32495 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:47994:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:10:16.209971 32495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:16.211530 32679 tablet_service.cc:1467] Processing CreateTablet for tablet ca2883135aab4262a42ce050eb4bea4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2278fc89f5e34974a09f205e2f91c549]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:10:16.211530 409 tablet_service.cc:1467] Processing CreateTablet for tablet ca2883135aab4262a42ce050eb4bea4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2278fc89f5e34974a09f205e2f91c549]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:10:16.211580 599 tablet_service.cc:1467] Processing CreateTablet for tablet ca2883135aab4262a42ce050eb4bea4a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2278fc89f5e34974a09f205e2f91c549]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:10:16.211704 409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca2883135aab4262a42ce050eb4bea4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:16.211706 32679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca2883135aab4262a42ce050eb4bea4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:16.211776 599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca2883135aab4262a42ce050eb4bea4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:16.213064 718 tablet_bootstrap.cc:492] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: Bootstrap starting.
I20260808 07:10:16.213192 717 tablet_bootstrap.cc:492] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: Bootstrap starting.
I20260808 07:10:16.213192 720 tablet_bootstrap.cc:492] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: Bootstrap starting.
I20260808 07:10:16.213529 720 tablet_bootstrap.cc:654] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:16.213527 718 tablet_bootstrap.cc:654] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:16.213588 717 tablet_bootstrap.cc:654] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:16.214162 720 tablet_bootstrap.cc:492] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: No bootstrap required, opened a new log
I20260808 07:10:16.214226 720 ts_tablet_manager.cc:1397] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:16.214257 717 tablet_bootstrap.cc:492] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: No bootstrap required, opened a new log
I20260808 07:10:16.214265 718 tablet_bootstrap.cc:492] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: No bootstrap required, opened a new log
I20260808 07:10:16.214314 717 ts_tablet_manager.cc:1397] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:16.214316 718 ts_tablet_manager.cc:1397] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:16.214359 720 raft_consensus.cc:348] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.214440 720 raft_consensus.cc:374] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:16.214469 720 raft_consensus.cc:729] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d2cef3d2e54520b0a4da67cf2be46f, State: Initialized, Role: FOLLOWER
I20260808 07:10:16.214458 717 raft_consensus.cc:348] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.214469 718 raft_consensus.cc:348] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.214533 717 raft_consensus.cc:374] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:16.214538 718 raft_consensus.cc:374] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:16.214517 720 consensus_queue.cc:260] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.214560 717 raft_consensus.cc:729] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 031b591d722d4083a83098d237a43db5, State: Initialized, Role: FOLLOWER
I20260808 07:10:16.214562 718 raft_consensus.cc:729] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf133e6848f64edb89f28494e17d2ff5, State: Initialized, Role: FOLLOWER
I20260808 07:10:16.214617 720 ts_tablet_manager.cc:1428] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:16.214610 718 consensus_queue.cc:260] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.214607 717 consensus_queue.cc:260] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.214697 718 ts_tablet_manager.cc:1428] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:16.214702 717 ts_tablet_manager.cc:1428] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:16.214738 731 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.214744 726 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.214833 731 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:16.214834 726 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:16.214946 725 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.215036 725 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:16.439400 726 raft_consensus.cc:482] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:16.439505 726 raft_consensus.cc:504] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.439869 726 leader_election.cc:288] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 031b591d722d4083a83098d237a43db5 (127.30.54.65:38619), 67d2cef3d2e54520b0a4da67cf2be46f (127.30.54.66:46267)
I20260808 07:10:16.443022 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.440257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:50577 (local address 127.30.54.65:38619)
0808 07:10:16.440312 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.440315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.440363 (+ 48us) server_negotiation.cc:407] Connection header received
0808 07:10:16.440439 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.440440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.440454 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.440521 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:16.440688 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.441182 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.442258 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.442613 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.442695 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.442706 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.442892 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.442930 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:10:16.443038 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.440166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:48271 (local address 127.30.54.66:46267)
0808 07:10:16.440213 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.440215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.440452 (+ 237us) server_negotiation.cc:407] Connection header received
0808 07:10:16.440523 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.440524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.440535 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.440605 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:16.440779 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.441305 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.442268 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.442648 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.442729 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.442738 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.442927 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.442968 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:10:16.443058 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.440100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:38619 (local address 127.30.54.67:50577)
0808 07:10:16.440292 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.440304 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.440371 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.440526 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.440527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.440529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:16.440612 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.440614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.441260 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.441261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.442167 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.442169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.442717 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.442718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.442798 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.442801 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.442886 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.442929 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260808 07:10:16.443066 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.440100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46267 (local address 127.30.54.67:48271)
0808 07:10:16.440371 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.440381 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.440465 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.440607 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.440608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.440610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:16.440703 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.440705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.441387 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.441388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.442177 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.442179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.442729 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.442730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.442855 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.442860 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.442908 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.442951 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":157,"thread_start_us":85,"threads_started":1}
I20260808 07:10:16.443318 32703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca2883135aab4262a42ce050eb4bea4a" candidate_uuid: "bf133e6848f64edb89f28494e17d2ff5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031b591d722d4083a83098d237a43db5" is_pre_election: true
I20260808 07:10:16.443383 434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca2883135aab4262a42ce050eb4bea4a" candidate_uuid: "bf133e6848f64edb89f28494e17d2ff5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" is_pre_election: true
I20260808 07:10:16.443429 32703 raft_consensus.cc:2454] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf133e6848f64edb89f28494e17d2ff5 in term 0.
I20260808 07:10:16.443476 434 raft_consensus.cc:2454] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf133e6848f64edb89f28494e17d2ff5 in term 0.
I20260808 07:10:16.443755 539 leader_election.cc:302] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031b591d722d4083a83098d237a43db5, bf133e6848f64edb89f28494e17d2ff5; no voters:
I20260808 07:10:16.443902 726 raft_consensus.cc:2781] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:16.443949 726 raft_consensus.cc:482] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:16.443972 726 raft_consensus.cc:3037] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:16.444715 726 raft_consensus.cc:504] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.444854 726 leader_election.cc:288] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [CANDIDATE]: Term 1 election: Requested vote from peers 031b591d722d4083a83098d237a43db5 (127.30.54.65:38619), 67d2cef3d2e54520b0a4da67cf2be46f (127.30.54.66:46267)
I20260808 07:10:16.445024 32703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca2883135aab4262a42ce050eb4bea4a" candidate_uuid: "bf133e6848f64edb89f28494e17d2ff5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "031b591d722d4083a83098d237a43db5"
I20260808 07:10:16.445079 434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca2883135aab4262a42ce050eb4bea4a" candidate_uuid: "bf133e6848f64edb89f28494e17d2ff5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67d2cef3d2e54520b0a4da67cf2be46f"
I20260808 07:10:16.445122 32703 raft_consensus.cc:3037] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:16.445154 434 raft_consensus.cc:3037] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:16.445912 32703 raft_consensus.cc:2454] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf133e6848f64edb89f28494e17d2ff5 in term 1.
I20260808 07:10:16.445994 434 raft_consensus.cc:2454] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf133e6848f64edb89f28494e17d2ff5 in term 1.
I20260808 07:10:16.446102 539 leader_election.cc:302] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 031b591d722d4083a83098d237a43db5, bf133e6848f64edb89f28494e17d2ff5; no voters:
I20260808 07:10:16.446188 726 raft_consensus.cc:2781] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:16.446324 726 raft_consensus.cc:686] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [term 1 LEADER]: Becoming Leader. State: Replica: bf133e6848f64edb89f28494e17d2ff5, State: Running, Role: LEADER
I20260808 07:10:16.446413 726 consensus_queue.cc:237] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } }
I20260808 07:10:16.446751 780 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: TxnStatusTablet state changed. Reason: New leader bf133e6848f64edb89f28494e17d2ff5. Latest consensus state: current_term: 1 leader_uuid: "bf133e6848f64edb89f28494e17d2ff5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.446866 780 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:10:16.447041 783 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:10:16.447158 783 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:10:16.447199 32493 catalog_manager.cc:5495] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf133e6848f64edb89f28494e17d2ff5 (127.30.54.67). New cstate: current_term: 1 leader_uuid: "bf133e6848f64edb89f28494e17d2ff5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } health_report { overall_health: HEALTHY } } }
I20260808 07:10:16.497594 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.495102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.30.54.67:36109)
0808 07:10:16.495165 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.495168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.495201 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:10:16.495291 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.495292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.495312 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.495387 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:16.495542 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.496022 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.496799 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.497004 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.497076 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.497263 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:16.497352 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:16.497429 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.497497 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.497517 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:10:16.497602 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.495036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:36109 (local address 127.0.0.1:48214)
0808 07:10:16.495152 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.495164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.495221 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.495394 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.495395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.495397 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:10:16.495458 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.495460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.496117 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.496119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.496714 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.496716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.497075 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.497076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.497137 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.497164 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:10:16.497423 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:10:16.497434 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.497480 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.497532 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 07:10:16.498826 32703 raft_consensus.cc:1261] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5 [term 1 FOLLOWER]: Refusing update from remote peer bf133e6848f64edb89f28494e17d2ff5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:16.498893 434 raft_consensus.cc:1261] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 FOLLOWER]: Refusing update from remote peer bf133e6848f64edb89f28494e17d2ff5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:16.499125 780 consensus_queue.cc:1035] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:16.499249 726 consensus_queue.cc:1035] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:16.499908 731 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: TxnStatusTablet state changed. Reason: New leader bf133e6848f64edb89f28494e17d2ff5. Latest consensus state: current_term: 1 leader_uuid: "bf133e6848f64edb89f28494e17d2ff5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.500013 731 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:16.500252 725 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: TxnStatusTablet state changed. Reason: New leader bf133e6848f64edb89f28494e17d2ff5. Latest consensus state: current_term: 1 leader_uuid: "bf133e6848f64edb89f28494e17d2ff5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.500348 725 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:16.500532 726 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf133e6848f64edb89f28494e17d2ff5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.500624 726 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:10:16.500684 737 mvcc.cc:205] Tried to move back new op lower bound from 7316164675576807424 to 7316164675364990976. Current Snapshot: MvccSnapshot[applied={T|T < 7316164675576807424}]
I20260808 07:10:16.500968 725 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf133e6848f64edb89f28494e17d2ff5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.501060 725 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P 031b591d722d4083a83098d237a43db5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:16.502651 780 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf133e6848f64edb89f28494e17d2ff5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.502736 780 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P bf133e6848f64edb89f28494e17d2ff5: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:10:16.503700 731 tablet_replica.cc:439] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf133e6848f64edb89f28494e17d2ff5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "031b591d722d4083a83098d237a43db5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38619 } } peers { permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 } } peers { permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 } } }
I20260808 07:10:16.503793 731 tablet_replica.cc:442] T ca2883135aab4262a42ce050eb4bea4a P 67d2cef3d2e54520b0a4da67cf2be46f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:16.505142 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.502348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49444 (local address 127.30.54.65:38619)
0808 07:10:16.502384 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.502386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.502472 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:10:16.502553 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.502553 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.502568 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.502652 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:16.502779 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.503403 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.504340 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.504614 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.504676 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.504816 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:16.504917 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:16.504986 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.505042 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.505062 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:10:16.510699 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.507495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:38611 (local address 127.30.54.67:36109)
0808 07:10:16.507558 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.507560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.507644 (+ 84us) server_negotiation.cc:407] Connection header received
0808 07:10:16.507715 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.507715 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.507722 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.507762 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:16.507908 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.508379 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.509985 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.510162 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.510213 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.510424 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:16.510448 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:16.510488 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.510596 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.510620 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:10:16.510744 791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.507417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:36109 (local address 127.30.54.65:38611)
0808 07:10:16.507585 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.507596 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.507660 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.507774 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.507775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.507776 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:10:16.507847 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.507849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.508460 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.508462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.509902 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.509906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.510239 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.510240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.510322 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.510338 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:10:16.510510 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:10:16.510512 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.510591 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.510633 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260808 07:10:16.513013 628 raft_consensus.cc:1261] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Refusing update from remote peer 031b591d722d4083a83098d237a43db5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:16.513052 434 raft_consensus.cc:1261] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 FOLLOWER]: Refusing update from remote peer 031b591d722d4083a83098d237a43db5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:16.513280 725 consensus_queue.cc:1035] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67d2cef3d2e54520b0a4da67cf2be46f" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:16.513406 762 consensus_queue.cc:1035] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf133e6848f64edb89f28494e17d2ff5" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 36109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:16.528460 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.526214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:54659 (local address 127.30.54.65:38619)
0808 07:10:16.526257 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.526259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.526266 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:10:16.526288 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.526289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.526297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.526352 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:16.526523 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.527039 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.527774 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.527980 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.528050 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.528219 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:16.528251 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:16.528313 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.528374 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.528392 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:16.528687 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.526000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:38619 (local address 127.30.54.67:54659)
0808 07:10:16.526158 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.526173 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:16.526220 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:16.526354 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:16.526355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:16.526357 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:10:16.526450 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.526452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.527119 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.527120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.527693 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:16.527695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.528069 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.528071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:16.528134 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.528146 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:10:16.528315 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:10:16.528317 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:16.528371 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:10:16.528419 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260808 07:10:16.539423 32494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:16.539515 32494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:16.540759 32494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8dd7193124741d5bae066d33f741609: Sending DeleteTablet for 3 replicas of tablet ded50494d9fc4732a3ce1797c24da778
I20260808 07:10:16.541026 409 tablet_service.cc:1514] Processing DeleteTablet for tablet ded50494d9fc4732a3ce1797c24da778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:16 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:58878
I20260808 07:10:16.541115 752 tablet_replica.cc:330] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: stopping tablet replica
I20260808 07:10:16.541141 32679 tablet_service.cc:1514] Processing DeleteTablet for tablet ded50494d9fc4732a3ce1797c24da778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:16 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:49434
I20260808 07:10:16.541172 752 raft_consensus.cc:2229] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:16.541208 752 raft_consensus.cc:2258] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:16.541283 747 tablet_replica.cc:330] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: stopping tablet replica
I20260808 07:10:16.541337 747 raft_consensus.cc:2229] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:16.541419 747 raft_consensus.cc:2258] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:16.541791 752 ts_tablet_manager.cc:1905] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:16.543603 747 ts_tablet_manager.cc:1905] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:16.545120 747 ts_tablet_manager.cc:1918] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:10:16.545185 747 log.cc:1196] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/ded50494d9fc4732a3ce1797c24da778
I20260808 07:10:16.545418 747 ts_tablet_manager.cc:1939] T ded50494d9fc4732a3ce1797c24da778 P 031b591d722d4083a83098d237a43db5: Deleting consensus metadata
I20260808 07:10:16.545722 32479 catalog_manager.cc:4841] TS 031b591d722d4083a83098d237a43db5 (127.30.54.65:38619): tablet ded50494d9fc4732a3ce1797c24da778 (table test-table [id=5a8cd1a2b84440e68ab9ca16fc0e0c6e]) successfully deleted
I20260808 07:10:16.545873 599 tablet_service.cc:1514] Processing DeleteTablet for tablet ded50494d9fc4732a3ce1797c24da778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:16 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48198
I20260808 07:10:16.545971 748 tablet_replica.cc:330] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: stopping tablet replica
I20260808 07:10:16.546044 748 raft_consensus.cc:2229] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:16.546048 752 ts_tablet_manager.cc:1918] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:10:16.546088 748 raft_consensus.cc:2258] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:16.546100 752 log.cc:1196] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/ded50494d9fc4732a3ce1797c24da778
I20260808 07:10:16.546310 752 ts_tablet_manager.cc:1939] T ded50494d9fc4732a3ce1797c24da778 P 67d2cef3d2e54520b0a4da67cf2be46f: Deleting consensus metadata
I20260808 07:10:16.546703 32475 catalog_manager.cc:4841] TS 67d2cef3d2e54520b0a4da67cf2be46f (127.30.54.66:46267): tablet ded50494d9fc4732a3ce1797c24da778 (table test-table [id=5a8cd1a2b84440e68ab9ca16fc0e0c6e]) successfully deleted
I20260808 07:10:16.546978 748 ts_tablet_manager.cc:1905] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:16.548960 748 ts_tablet_manager.cc:1918] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:10:16.549026 748 log.cc:1196] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/ded50494d9fc4732a3ce1797c24da778
I20260808 07:10:16.549247 748 ts_tablet_manager.cc:1939] T ded50494d9fc4732a3ce1797c24da778 P bf133e6848f64edb89f28494e17d2ff5: Deleting consensus metadata
I20260808 07:10:16.549547 32475 catalog_manager.cc:4841] TS bf133e6848f64edb89f28494e17d2ff5 (127.30.54.67:36109): tablet ded50494d9fc4732a3ce1797c24da778 (table test-table [id=5a8cd1a2b84440e68ab9ca16fc0e0c6e]) successfully deleted
Aug 08 07:10:16 dist-test-slave-znh6 krb5kdc[32431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173015, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:16.554664 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.543086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.30.54.65:38619)
0808 07:10:16.543125 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.543128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.543402 (+ 274us) server_negotiation.cc:407] Connection header received
0808 07:10:16.543498 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.543500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.543509 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.543539 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:16.544386 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.545073 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.546563 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.546885 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.546929 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.550888 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:16.550897 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:16.550898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:16.550905 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:16.553263 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:16.553666 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:16.553669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:16.553670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:16.553756 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:16.553981 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:16.553982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:16.553983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:16.554214 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:16.554388 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.554475 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.554562 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:10:16.555347 32754 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49452
I20260808 07:10:16.563753 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.556338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.30.54.126:36565)
0808 07:10:16.556384 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:16.556386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:16.556454 (+ 68us) server_negotiation.cc:407] Connection header received
0808 07:10:16.556522 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:16.556523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:16.556534 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:16.556607 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:16.557178 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.557924 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.559087 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:16.559379 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:16.559454 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:16.560796 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:16.560797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:16.560798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:16.560806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:16.562448 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:16.562881 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:16.562882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:16.562883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:16.562947 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:16.563173 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:16.563175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:16.563175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:16.563341 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:16.563446 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:16.563580 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 07:10:16.563663 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 07:10:16.563941 32494 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48010
I20260808 07:10:16.564698 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 32597
I20260808 07:10:16.579371 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 318
I20260808 07:10:16.589531 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 517
I20260808 07:10:16.598709 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 32449
2026-08-08T07:10:16Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2878 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:16 dist-test-slave-znh6 krb5kdc[808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:16 dist-test-slave-znh6 krb5kdc[808](info): set up 2 sockets
Aug 08 07:10:16 dist-test-slave-znh6 krb5kdc[808](info): commencing operation
krb5kdc: starting...
W20260808 07:10:17.818315 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:17 dist-test-slave-znh6 krb5kdc[808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:17.937471 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39041
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:39041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:18.018761 828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:18.019058 828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:18.019083 828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:18.020336 828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:18.020380 828 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:10:18.020393 828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:18.020406 828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:18.020455 828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:18.020473 828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:18.022220 828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:39041
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:18.022467 828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:18.022691 828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:18.027676 837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:18.027679 834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:18.027647 835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:18.027793 828 server_base.cc:1034] running on GCE node
I20260808 07:10:18.028023 828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:18.028319 828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:18.029480 828 hybrid_clock.cc:648] HybridClock initialized: now 1786173018029461 us; error 37 us; skew 500 ppm
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:18.032249 828 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:18.032617 828 webserver.cc:460] Webserver started at http://127.30.54.126:36835/ using document root <none> and password file <none>
I20260808 07:10:18.032792 828 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:18.032829 828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:18.032923 828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:18.033890 828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "12bbe7740c944c26a0cc62ef86ed8e69"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "1f991c54f5bd6028367b470f46847480"
server_key_iv: "d241ed68cdd4d48888a1ac1c68bb2b7b"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.034264 828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "12bbe7740c944c26a0cc62ef86ed8e69"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "1f991c54f5bd6028367b470f46847480"
server_key_iv: "d241ed68cdd4d48888a1ac1c68bb2b7b"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.035861 828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:18.036545 848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.036720 828 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:18.036789 828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "12bbe7740c944c26a0cc62ef86ed8e69"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "1f991c54f5bd6028367b470f46847480"
server_key_iv: "d241ed68cdd4d48888a1ac1c68bb2b7b"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.036840 828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:18.045151 828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:18.047303 828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:18.047415 828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:18.052083 828 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:39041
I20260808 07:10:18.052166 935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:39041 every 8 connection(s)
I20260808 07:10:18.052467 828 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:18.053160 936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.055419 936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69: Bootstrap starting.
I20260808 07:10:18.055996 936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.056275 936 log.cc:824] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:18.056924 936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69: No bootstrap required, opened a new log
I20260808 07:10:18.059034 936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bbe7740c944c26a0cc62ef86ed8e69" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39041 } }
I20260808 07:10:18.059163 936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.059207 936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12bbe7740c944c26a0cc62ef86ed8e69, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.059379 936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bbe7740c944c26a0cc62ef86ed8e69" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39041 } }
I20260808 07:10:18.059463 936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:18.059515 936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:18.059566 936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.060075 936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bbe7740c944c26a0cc62ef86ed8e69" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39041 } }
I20260808 07:10:18.060185 936 leader_election.cc:302] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12bbe7740c944c26a0cc62ef86ed8e69; no voters:
I20260808 07:10:18.060362 936 leader_election.cc:288] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:18.060410 942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:18.060719 936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:18.060715 942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [term 1 LEADER]: Becoming Leader. State: Replica: 12bbe7740c944c26a0cc62ef86ed8e69, State: Running, Role: LEADER
I20260808 07:10:18.060847 942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bbe7740c944c26a0cc62ef86ed8e69" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39041 } }
I20260808 07:10:18.061246 943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12bbe7740c944c26a0cc62ef86ed8e69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bbe7740c944c26a0cc62ef86ed8e69" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39041 } } }
I20260808 07:10:18.061250 945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12bbe7740c944c26a0cc62ef86ed8e69. Latest consensus state: current_term: 1 leader_uuid: "12bbe7740c944c26a0cc62ef86ed8e69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12bbe7740c944c26a0cc62ef86ed8e69" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39041 } } }
I20260808 07:10:18.061367 945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:18.061363 943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:18.061582 956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:18.062135 956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:18.062258 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 828
I20260808 07:10:18.062353 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:18.062507 30937 external_mini_cluster.cc:1428] Setting key 35b3367edf974a021c516d256cae5eaa
I20260808 07:10:18.066408 956 catalog_manager.cc:1347] Generated new cluster ID: c395e04cc5314f28b59e6f9e8146e036
I20260808 07:10:18.066461 956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:18.075093 956 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:18.075466 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.062938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.30.54.126:39041)
0808 07:10:18.064500 (+ 1562us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.064509 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.064571 (+ 62us) server_negotiation.cc:407] Connection header received
0808 07:10:18.064710 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.064717 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.064796 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.064930 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:18.065841 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.067344 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.068388 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.069054 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.069167 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.071896 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:18.071900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:18.071900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:18.071912 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:18.073800 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.074214 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.074217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.074217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.074315 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.074535 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.074536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.074536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.074803 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:18.075006 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.075097 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.075243 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1519,"thread_start_us":73,"threads_started":1}
I20260808 07:10:18.075910 956 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:18.080440 956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69: Generated new TSK 0
I20260808 07:10:18.080624 956 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:10:18.109280 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39041
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:18.189967 973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:18.190160 973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:18.190183 973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:18.191354 973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:18.191393 973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:18.191414 973 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:10:18.191462 973 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:10:18.191490 973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:18.192890 973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.54.126:39041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:18.193115 973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:18.193383 973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:18.198341 985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:18.198388 981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:18.198443 973 server_base.cc:1034] running on GCE node
W20260808 07:10:18.198366 990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:18.198724 973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:18.198993 973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:18.200135 973 hybrid_clock.cc:648] HybridClock initialized: now 1786173018200111 us; error 42 us; skew 500 ppm
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:18.202654 973 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:10:18.203127 973 webserver.cc:460] Webserver started at http://127.30.54.65:46121/ using document root <none> and password file <none>
I20260808 07:10:18.203346 973 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:18.203392 973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:18.203522 973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:18.204514 973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "8fd4d830d298407093afc226147ee7d4"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "e9141f8e199c59e04f6f511d90787eb2"
server_key_iv: "d4ead03ae6e367aebd71657f45bb5a71"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.204818 973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "8fd4d830d298407093afc226147ee7d4"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "e9141f8e199c59e04f6f511d90787eb2"
server_key_iv: "d4ead03ae6e367aebd71657f45bb5a71"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.206110 973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:18.206913 1001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.207095 973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:18.207180 973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "8fd4d830d298407093afc226147ee7d4"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "e9141f8e199c59e04f6f511d90787eb2"
server_key_iv: "d4ead03ae6e367aebd71657f45bb5a71"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.207230 973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:18.224850 973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:18.226567 973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:18.226673 973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:18.227195 973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:18.227236 973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.227257 973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:18.227269 973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.234268 973 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:40967
I20260808 07:10:18.234354 1163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:40967 every 8 connection(s)
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:18.234659 973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:18.237723 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.227685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:52049 (local address 127.30.54.126:39041)
0808 07:10:18.227767 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.227770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.228016 (+ 246us) server_negotiation.cc:407] Connection header received
0808 07:10:18.228571 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.228573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.228587 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.228651 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:18.229574 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.230445 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.232111 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.232337 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.232394 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.235126 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:18.235128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:18.235129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:18.235137 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:18.236327 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.236739 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.236741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.236741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.236804 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.237063 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.237064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.237065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.237184 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:18.237262 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.237547 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.237635 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:10:18.237891 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.227644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39041 (local address 127.30.54.65:52049)
0808 07:10:18.227912 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.227939 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.228457 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.228772 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.228777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.229162 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:18.229481 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.229488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.230556 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.230557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.232017 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.232021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.232418 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.232420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.232587 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.233186 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:18.233197 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:18.235024 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:18.236422 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.236428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.236431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.236651 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.236870 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.236871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.236871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.236986 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.237251 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:18.237258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:18.237392 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.237520 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.237672 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":114,"threads_started":1}
I20260808 07:10:18.238402 1164 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39041
I20260808 07:10:18.238488 1164 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:18.238678 1164 heartbeater.cc:511] Master 127.30.54.126:39041 requested a full tablet report, sending...
I20260808 07:10:18.239261 868 ts_manager.cc:194] Registered new tserver with Master: 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967)
I20260808 07:10:18.240206 868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:52049
I20260808 07:10:18.244215 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 973
I20260808 07:10:18.244314 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:18.244539 30937 external_mini_cluster.cc:1428] Setting key c33e35a433b673ca65457b37ba525498
I20260808 07:10:18.245041 1170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.239515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39041 (local address 127.30.54.65:52379)
0808 07:10:18.239659 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.239670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.239732 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.239877 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.239878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.240056 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:18.240111 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.240112 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.240936 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.240937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.241465 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.241467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.241772 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.241773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.241834 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.242258 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:18.242263 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:18.242532 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:18.243893 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.243894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.243895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.244050 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.244326 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.244328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.244328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.244407 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.244720 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:18.244721 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:18.244768 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.244885 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.244981 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260808 07:10:18.245107 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.239610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:52379 (local address 127.30.54.126:39041)
0808 07:10:18.239648 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.239649 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.239725 (+ 76us) server_negotiation.cc:407] Connection header received
0808 07:10:18.239786 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.239787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.239794 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.239863 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:18.240176 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.240840 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.241538 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.241709 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.241757 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.242631 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:18.242633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:18.242633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:18.242639 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:18.243814 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.244123 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.244124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.244124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.244188 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.244477 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.244478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.244478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.244615 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:18.244687 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.244933 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.245017 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:10:18.269891 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39041
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:18.349692 1175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:18.349989 1175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:18.350029 1175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:18.351182 1175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:18.351222 1175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:18.351243 1175 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:10:18.351254 1175 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:10:18.351271 1175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:18.352653 1175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.54.126:39041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:18.352836 1175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:18.353004 1175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:18.357849 1183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:18.357820 1182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:18.357824 1187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:18.357976 1175 server_base.cc:1034] running on GCE node
I20260808 07:10:18.358248 1175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:18.358484 1175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:18.359700 1175 hybrid_clock.cc:648] HybridClock initialized: now 1786173018359683 us; error 30 us; skew 500 ppm
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:18.362367 1175 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:10:18.362852 1175 webserver.cc:460] Webserver started at http://127.30.54.66:33175/ using document root <none> and password file <none>
I20260808 07:10:18.363058 1175 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:18.363101 1175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:18.363184 1175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:18.364182 1175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "12cce22d691f432bacd9f76aeead2a44"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "392c423128163a057b3db1948fbdaca9"
server_key_iv: "e9a61c5606ffce8d5a096898889ed7f7"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.364488 1175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "12cce22d691f432bacd9f76aeead2a44"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "392c423128163a057b3db1948fbdaca9"
server_key_iv: "e9a61c5606ffce8d5a096898889ed7f7"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.365743 1175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:18.366417 1195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.366587 1175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:18.366691 1175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "12cce22d691f432bacd9f76aeead2a44"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "392c423128163a057b3db1948fbdaca9"
server_key_iv: "e9a61c5606ffce8d5a096898889ed7f7"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.366765 1175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:18.381883 1175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:18.384255 1175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:18.384370 1175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:18.384792 1175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:18.384860 1175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.384903 1175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:18.384932 1175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:18.391971 1175 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:46617
I20260808 07:10:18.392037 1365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:46617 every 8 connection(s)
I20260808 07:10:18.392328 1175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:18.394490 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 1175
I20260808 07:10:18.394582 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:18.394845 30937 external_mini_cluster.cc:1428] Setting key 1306681b023c102f51179bbea5978683
I20260808 07:10:18.395385 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.385262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:48281 (local address 127.30.54.126:39041)
0808 07:10:18.385336 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.385338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.385600 (+ 262us) server_negotiation.cc:407] Connection header received
0808 07:10:18.386134 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.386135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.386150 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.386235 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:18.387054 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.387874 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.389440 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.389804 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.389879 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.392427 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:18.392428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:18.392429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:18.392436 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:18.393761 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.394164 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.394166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.394166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.394224 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.394537 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.394538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.394539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.394700 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:18.394794 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.395216 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.395298 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:10:18.395500 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.385225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39041 (local address 127.30.54.66:48281)
0808 07:10:18.385498 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.385525 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.385990 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.386313 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.386317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.386674 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:18.386955 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.386959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.387979 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.387980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.389342 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.389345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.389901 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.389903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.390059 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.390585 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:18.390595 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:18.392331 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:18.393858 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.393864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.393866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.394085 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.394297 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.394297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.394298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.394445 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.394874 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:18.394880 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:18.395035 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.395160 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.395307 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260808 07:10:18.396082 1367 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39041
I20260808 07:10:18.396174 1367 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:18.396400 1367 heartbeater.cc:511] Master 127.30.54.126:39041 requested a full tablet report, sending...
I20260808 07:10:18.396766 869 ts_manager.cc:194] Registered new tserver with Master: 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617)
I20260808 07:10:18.397286 869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:48281
I20260808 07:10:18.404747 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.397035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39041 (local address 127.30.54.66:57297)
0808 07:10:18.397169 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.397179 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.397255 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.397421 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.397422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.397579 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:18.397625 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.397626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.399143 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.399144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.400033 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.400037 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.400613 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.400614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.400747 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.401197 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:18.401201 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:18.401476 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:18.403432 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.403433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.403434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.403639 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.403918 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.403919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.403919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.403995 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.404408 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:18.404409 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:18.404456 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.404529 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.404628 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260808 07:10:18.404779 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.397083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:57297 (local address 127.30.54.126:39041)
0808 07:10:18.397119 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.397121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.397242 (+ 121us) server_negotiation.cc:407] Connection header received
0808 07:10:18.397312 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.397313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.397323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.397369 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:18.397725 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.399029 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.400189 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.400537 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.400612 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.401559 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:18.401561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:18.401561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:18.401569 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:18.403290 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.403707 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.403709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.403709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.403810 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.404092 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.404093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.404094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.404303 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:18.404353 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.404582 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.404663 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:10:18.425979 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39041
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:18.507470 1379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:18.507649 1379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:18.507673 1379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:18.508714 1379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:18.508747 1379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:18.508767 1379 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:10:18.508780 1379 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:10:18.508796 1379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:18.510154 1379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.54.126:39041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:18.510327 1379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:18.510491 1379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:18.515321 1389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:18.515393 1387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:18.515534 1379 server_base.cc:1034] running on GCE node
W20260808 07:10:18.515403 1386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:18.515825 1379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:18.516062 1379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:18.517222 1379 hybrid_clock.cc:648] HybridClock initialized: now 1786173018517211 us; error 28 us; skew 500 ppm
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:18.519717 1379 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:10:18.520174 1379 webserver.cc:460] Webserver started at http://127.30.54.67:33981/ using document root <none> and password file <none>
I20260808 07:10:18.520406 1379 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:18.520475 1379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:18.520622 1379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:18.521589 1379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "2ca97944e79a8cd9152d93fff478dd3c"
server_key_iv: "5637785df192a8536c83d24228c842a6"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.522018 1379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "2ca97944e79a8cd9152d93fff478dd3c"
server_key_iv: "5637785df192a8536c83d24228c842a6"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.523381 1379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:10:18.524076 1398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.524238 1379 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:18.524338 1379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e"
format_stamp: "Formatted at 2026-08-08 07:10:18 on dist-test-slave-znh6"
server_key: "2ca97944e79a8cd9152d93fff478dd3c"
server_key_iv: "5637785df192a8536c83d24228c842a6"
server_key_version: "encryptionkey@0"
I20260808 07:10:18.524415 1379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:18.540858 1379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:18.543062 1379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:18.543224 1379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:18.543819 1379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:18.543869 1379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.543910 1379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:18.543943 1379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:18.551312 1379 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:46137
I20260808 07:10:18.551658 1379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:18.551700 1562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:46137 every 8 connection(s)
I20260808 07:10:18.553830 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.544221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:56255 (local address 127.30.54.126:39041)
0808 07:10:18.544277 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.544280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.544489 (+ 209us) server_negotiation.cc:407] Connection header received
0808 07:10:18.544926 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.544927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.544942 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.545026 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:18.545642 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.546435 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.547602 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.547818 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.547898 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.551190 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:18.551191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:18.551192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:18.551200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:18.552753 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.553107 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.553108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.553108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.553159 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.553360 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.553360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.553361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.553458 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:18.553508 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.553711 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.553769 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:10:18.554033 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.544195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39041 (local address 127.30.54.67:56255)
0808 07:10:18.544351 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.544389 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.544831 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.545075 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.545077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.545312 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:18.545530 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.545536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.546536 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.546537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.547509 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.547512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.547911 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.547912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.548047 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.548590 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:18.548600 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:18.551071 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:18.552828 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.552833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.552834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.553041 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.553217 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.553218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.553218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.553309 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.553518 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:18.553521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:18.553621 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.553698 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.553804 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260808 07:10:18.554539 1563 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39041
I20260808 07:10:18.554622 1563 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:18.554793 1563 heartbeater.cc:511] Master 127.30.54.126:39041 requested a full tablet report, sending...
I20260808 07:10:18.555256 868 ts_manager.cc:194] Registered new tserver with Master: a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137)
I20260808 07:10:18.555706 868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:56255
I20260808 07:10:18.560460 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.555566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39041 (local address 127.30.54.67:58701)
0808 07:10:18.555711 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.555723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.555776 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.555970 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.555971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.556175 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:18.556234 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.556236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.556841 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.556842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.557347 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.557348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.557646 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.557646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.557707 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.558111 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:18.558116 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:18.558380 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:18.559674 (+ 1294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.559675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.559675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.559827 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.559993 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.559994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.559994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.560041 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.560201 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:18.560201 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:18.560234 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.560315 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.560400 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260808 07:10:18.560495 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.555649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:58701 (local address 127.30.54.126:39041)
0808 07:10:18.555837 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.555839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.555845 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:10:18.555862 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.555862 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.555870 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.555949 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:18.556316 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.556747 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.557414 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.557585 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.557633 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.558450 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:18.558451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:18.558451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:18.558456 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:18.559607 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.559889 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.559890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.559890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.559936 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.560079 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.560079 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.560079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.560156 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:18.560199 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.560339 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.560404 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260808 07:10:18.561712 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 1379
I20260808 07:10:18.561816 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:18.562031 30937 external_mini_cluster.cc:1428] Setting key 0683536ecdb0a6f33f07b9d5de52f716
I20260808 07:10:18.562855 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:18.585312 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.578787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46158 (local address 127.30.54.126:39041)
0808 07:10:18.578876 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.578878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.578889 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:18.578950 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.578951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.578959 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.579009 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:18.579528 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.579999 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.580963 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.581182 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.581239 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.583220 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:18.583222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:18.583222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:18.583228 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:18.584334 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.584673 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.584674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.584674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.584732 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.584924 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.584925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.584925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.585031 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:18.585111 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.585191 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.585244 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 07:10:18.586683 868 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:18.588469 868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:18.594928 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.591025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:46137 (local address 127.0.0.1:52912)
0808 07:10:18.591204 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.591215 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.591269 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.591636 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.591637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.591641 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:18.591720 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.591722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.592575 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.592576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.593639 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.593641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.594574 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.594575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.594662 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.594676 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.594772 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.594848 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":18,"thread_start_us":79,"threads_started":1}
I20260808 07:10:18.594956 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.591237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.30.54.67:46137)
0808 07:10:18.591460 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.591464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.591474 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:18.591510 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.591512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.591527 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.591696 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:18.591782 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.592488 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.593728 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.594488 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.594573 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.594634 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.594796 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.594865 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":112,"threads_started":1}
I20260808 07:10:18.595371 1590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.590769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46617 (local address 127.0.0.1:58640)
0808 07:10:18.590928 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.590972 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.591040 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.591670 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.591670 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.591672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:18.591733 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.591735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.593095 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.593096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.594090 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.594093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.595024 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.595025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.595206 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.595211 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.595248 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.595292 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":35,"thread_start_us":111,"threads_started":1}
I20260808 07:10:18.595649 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.590769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40967 (local address 127.0.0.1:42994)
0808 07:10:18.591157 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.591166 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.591210 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.591633 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.591636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.591638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:18.591812 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.591814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.593113 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.593114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.594296 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.594299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.595436 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.595436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.595497 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.595500 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.595550 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.595586 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":29,"thread_start_us":120,"threads_started":1}
I20260808 07:10:18.595736 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.590908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58640 (local address 127.30.54.66:46617)
0808 07:10:18.591490 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.591495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.591505 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:18.591545 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.591547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.591564 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.591812 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:18.591886 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.592990 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.594179 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.594923 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.595492 (+ 569us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.595564 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.595589 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.595630 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":90,"threads_started":1}
I20260808 07:10:18.595738 1470 tablet_service.cc:1467] Processing CreateTablet for tablet 69081290170b4d5689b19cf899223345 (DEFAULT_TABLE table=test-table [id=605cf118923e4bd58ad3449f54c96e82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:18.595945 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.591017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.30.54.65:40967)
0808 07:10:18.591302 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.591306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.591314 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:10:18.591365 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.591367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.591385 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.591550 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:18.591894 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.593023 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.594391 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.595273 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.595335 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.595387 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.595792 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.595843 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":88,"threads_started":1}
I20260808 07:10:18.596161 1470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69081290170b4d5689b19cf899223345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.596544 1071 tablet_service.cc:1467] Processing CreateTablet for tablet 69081290170b4d5689b19cf899223345 (DEFAULT_TABLE table=test-table [id=605cf118923e4bd58ad3449f54c96e82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:18.596879 1071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69081290170b4d5689b19cf899223345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.598704 1596 tablet_bootstrap.cc:492] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: Bootstrap starting.
I20260808 07:10:18.599702 1596 tablet_bootstrap.cc:654] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.599637 1276 tablet_service.cc:1467] Processing CreateTablet for tablet 69081290170b4d5689b19cf899223345 (DEFAULT_TABLE table=test-table [id=605cf118923e4bd58ad3449f54c96e82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:18.599857 1599 tablet_bootstrap.cc:492] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: Bootstrap starting.
I20260808 07:10:18.599998 1276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69081290170b4d5689b19cf899223345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.600051 1596 log.cc:824] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:18.600504 1599 tablet_bootstrap.cc:654] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.600790 1599 log.cc:824] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:18.600848 1596 tablet_bootstrap.cc:492] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: No bootstrap required, opened a new log
I20260808 07:10:18.600916 1596 ts_tablet_manager.cc:1397] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:10:18.602998 1596 raft_consensus.cc:348] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } }
I20260808 07:10:18.603173 1596 raft_consensus.cc:374] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.603212 1596 raft_consensus.cc:729] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42cd1f61a6c40bebe01f11ad78e7b4e, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.603359 1596 consensus_queue.cc:260] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } }
I20260808 07:10:18.603600 1603 tablet_bootstrap.cc:492] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: Bootstrap starting.
I20260808 07:10:18.603681 1596 ts_tablet_manager.cc:1428] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:10:18.603816 1563 heartbeater.cc:503] Master 127.30.54.126:39041 was elected leader, sending a full tablet report...
I20260808 07:10:18.604206 1603 tablet_bootstrap.cc:654] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.604378 1599 tablet_bootstrap.cc:492] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: No bootstrap required, opened a new log
I20260808 07:10:18.604453 1599 ts_tablet_manager.cc:1397] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 07:10:18.604537 1603 log.cc:824] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:18.605262 1603 tablet_bootstrap.cc:492] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: No bootstrap required, opened a new log
I20260808 07:10:18.605315 1603 ts_tablet_manager.cc:1397] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:18.606070 1599 raft_consensus.cc:348] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } }
I20260808 07:10:18.606218 1599 raft_consensus.cc:374] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.606263 1599 raft_consensus.cc:729] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd4d830d298407093afc226147ee7d4, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.606379 1599 consensus_queue.cc:260] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } }
I20260808 07:10:18.606669 1599 ts_tablet_manager.cc:1428] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:10:18.606714 1164 heartbeater.cc:503] Master 127.30.54.126:39041 was elected leader, sending a full tablet report...
I20260808 07:10:18.606895 1603 raft_consensus.cc:348] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } }
I20260808 07:10:18.607044 1603 raft_consensus.cc:374] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.607086 1603 raft_consensus.cc:729] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12cce22d691f432bacd9f76aeead2a44, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.607206 1603 consensus_queue.cc:260] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } }
I20260808 07:10:18.607491 1367 heartbeater.cc:503] Master 127.30.54.126:39041 was elected leader, sending a full tablet report...
I20260808 07:10:18.607472 1603 ts_tablet_manager.cc:1428] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:18.617645 1605 raft_consensus.cc:482] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:18.617720 1605 raft_consensus.cc:504] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } }
I20260808 07:10:18.618098 1605 leader_election.cc:288] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617), 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967)
I20260808 07:10:18.623126 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.618477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:47433 (local address 127.30.54.65:40967)
0808 07:10:18.618585 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.618588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.618865 (+ 277us) server_negotiation.cc:407] Connection header received
0808 07:10:18.618936 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.618937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.618947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.619008 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:18.619141 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.620071 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.621686 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.622501 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.622707 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.622717 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.623023 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.623064 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 07:10:18.623126 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.618380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:33391 (local address 127.30.54.66:46617)
0808 07:10:18.618424 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.618426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.618453 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:10:18.618639 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.618640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.618649 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.618760 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:18.618960 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.619987 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.621686 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.622604 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.622712 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.622719 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.623023 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.623064 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:18.623124 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.618312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46617 (local address 127.30.54.67:33391)
0808 07:10:18.618381 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.618392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.618525 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.618781 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.618782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.618784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:18.618885 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.618887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.620061 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.620061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.621549 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.621557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.622715 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.622716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.622941 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.622944 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.622994 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.623046 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":6}
I20260808 07:10:18.623216 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.618312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40967 (local address 127.30.54.67:47433)
0808 07:10:18.618785 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.618796 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.618876 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.619020 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.619020 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.619022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:18.619075 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.619076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.620240 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.620240 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.621559 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.621561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.622724 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.622725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.622940 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.622944 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.622994 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.623082 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":258,"thread_start_us":115,"threads_started":1}
I20260808 07:10:18.623463 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69081290170b4d5689b19cf899223345" candidate_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12cce22d691f432bacd9f76aeead2a44" is_pre_election: true
I20260808 07:10:18.623564 1103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69081290170b4d5689b19cf899223345" candidate_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd4d830d298407093afc226147ee7d4" is_pre_election: true
I20260808 07:10:18.623615 1300 raft_consensus.cc:2454] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a42cd1f61a6c40bebe01f11ad78e7b4e in term 0.
I20260808 07:10:18.623711 1103 raft_consensus.cc:2454] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a42cd1f61a6c40bebe01f11ad78e7b4e in term 0.
I20260808 07:10:18.623855 1402 leader_election.cc:302] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12cce22d691f432bacd9f76aeead2a44, a42cd1f61a6c40bebe01f11ad78e7b4e; no voters:
I20260808 07:10:18.624028 1605 raft_consensus.cc:2781] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:18.624087 1605 raft_consensus.cc:482] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:18.624121 1605 raft_consensus.cc:3037] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.624758 1605 raft_consensus.cc:504] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } }
I20260808 07:10:18.624910 1605 leader_election.cc:288] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [CANDIDATE]: Term 1 election: Requested vote from peers 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617), 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967)
I20260808 07:10:18.625054 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69081290170b4d5689b19cf899223345" candidate_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12cce22d691f432bacd9f76aeead2a44"
I20260808 07:10:18.625066 1103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69081290170b4d5689b19cf899223345" candidate_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd4d830d298407093afc226147ee7d4"
I20260808 07:10:18.625145 1300 raft_consensus.cc:3037] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.625142 1103 raft_consensus.cc:3037] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.625766 1300 raft_consensus.cc:2454] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a42cd1f61a6c40bebe01f11ad78e7b4e in term 1.
I20260808 07:10:18.625815 1103 raft_consensus.cc:2454] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a42cd1f61a6c40bebe01f11ad78e7b4e in term 1.
I20260808 07:10:18.625954 1402 leader_election.cc:302] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12cce22d691f432bacd9f76aeead2a44, a42cd1f61a6c40bebe01f11ad78e7b4e; no voters:
I20260808 07:10:18.626113 1605 raft_consensus.cc:2781] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:18.626235 1605 raft_consensus.cc:686] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 LEADER]: Becoming Leader. State: Replica: a42cd1f61a6c40bebe01f11ad78e7b4e, State: Running, Role: LEADER
I20260808 07:10:18.626327 1605 consensus_queue.cc:237] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } }
I20260808 07:10:18.627082 868 catalog_manager.cc:5495] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e reported cstate change: term changed from 0 to 1, leader changed from <none> to a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67). New cstate: current_term: 1 leader_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } health_report { overall_health: UNKNOWN } } }
W20260808 07:10:18.642949 1368 tablet.cc:2369] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:10:18 dist-test-slave-znh6 krb5kdc[808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:18.662184 1625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.655494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39041 (local address 127.0.0.1:46162)
0808 07:10:18.655688 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.655704 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.655771 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.655904 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.655905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.656206 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:18.656256 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.656258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.656889 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.656889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.657418 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.657420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.657742 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.657743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.657786 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.658257 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:18.658262 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:18.659686 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:18.661104 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.661111 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.661113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.661358 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.661556 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:18.661557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:18.661557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:18.661650 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:18.661929 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:18.661935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:18.661974 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.662026 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.662112 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":132,"threads_started":1}
I20260808 07:10:18.662305 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.655601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46162 (local address 127.30.54.126:39041)
0808 07:10:18.655665 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.655667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.655772 (+ 105us) server_negotiation.cc:407] Connection header received
0808 07:10:18.655817 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.655817 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.655827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.655887 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:18.656323 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.656814 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.657494 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.657667 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.657717 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.659765 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:18.659767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:18.659767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:18.659773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:18.661036 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.661436 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.661437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.661437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.661494 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:18.661714 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:18.661714 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:18.661715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:18.661870 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:18.661923 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.662058 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.662160 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":38}
I20260808 07:10:18.664170 868 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:46162:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:10:18.664402 868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:18.666014 1276 tablet_service.cc:1467] Processing CreateTablet for tablet 105c1bd357114cf28eee19b47b3d082a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=520c7e769a9d4de1b7728ee90bf05e6c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:10:18.666061 1470 tablet_service.cc:1467] Processing CreateTablet for tablet 105c1bd357114cf28eee19b47b3d082a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=520c7e769a9d4de1b7728ee90bf05e6c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:10:18.666106 1071 tablet_service.cc:1467] Processing CreateTablet for tablet 105c1bd357114cf28eee19b47b3d082a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=520c7e769a9d4de1b7728ee90bf05e6c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:10:18.666209 1276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105c1bd357114cf28eee19b47b3d082a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.666224 1470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105c1bd357114cf28eee19b47b3d082a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.666244 1071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105c1bd357114cf28eee19b47b3d082a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.667773 1599 tablet_bootstrap.cc:492] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: Bootstrap starting.
I20260808 07:10:18.667775 1603 tablet_bootstrap.cc:492] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: Bootstrap starting.
I20260808 07:10:18.667930 1596 tablet_bootstrap.cc:492] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: Bootstrap starting.
I20260808 07:10:18.668388 1603 tablet_bootstrap.cc:654] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.668426 1596 tablet_bootstrap.cc:654] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.668505 1599 tablet_bootstrap.cc:654] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.669070 1603 tablet_bootstrap.cc:492] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: No bootstrap required, opened a new log
I20260808 07:10:18.669137 1603 ts_tablet_manager.cc:1397] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:18.669229 1596 tablet_bootstrap.cc:492] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: No bootstrap required, opened a new log
I20260808 07:10:18.669272 1596 ts_tablet_manager.cc:1397] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:10:18.669294 1603 raft_consensus.cc:348] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.669374 1603 raft_consensus.cc:374] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.669409 1603 raft_consensus.cc:729] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12cce22d691f432bacd9f76aeead2a44, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.669387 1596 raft_consensus.cc:348] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.669469 1596 raft_consensus.cc:374] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.669456 1603 consensus_queue.cc:260] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.669494 1596 raft_consensus.cc:729] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42cd1f61a6c40bebe01f11ad78e7b4e, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.669544 1603 ts_tablet_manager.cc:1428] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:18.669559 1596 consensus_queue.cc:260] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.669653 1596 ts_tablet_manager.cc:1428] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.669680 1612 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.669762 1612 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:18.669781 1605 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.669857 1605 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:18.670145 1599 tablet_bootstrap.cc:492] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: No bootstrap required, opened a new log
I20260808 07:10:18.670192 1599 ts_tablet_manager.cc:1397] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:18.670338 1599 raft_consensus.cc:348] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.670404 1599 raft_consensus.cc:374] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.670424 1599 raft_consensus.cc:729] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd4d830d298407093afc226147ee7d4, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.670469 1599 consensus_queue.cc:260] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.670581 1599 ts_tablet_manager.cc:1428] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.670652 1609 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.670733 1609 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 07:10:18.735646 1165 tablet.cc:2369] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:18.742909 1609 raft_consensus.cc:482] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:18.743008 1609 raft_consensus.cc:504] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.743345 1609 leader_election.cc:288] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617), a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137)
I20260808 07:10:18.746199 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.743589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:40457 (local address 127.30.54.66:46617)
0808 07:10:18.743653 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.743655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.743703 (+ 48us) server_negotiation.cc:407] Connection header received
0808 07:10:18.743779 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.743780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.743790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.743860 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:18.744037 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.744530 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.745482 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.745838 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.745918 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.745927 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.746099 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.746135 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:10:18.746223 1630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.743512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46617 (local address 127.30.54.65:40457)
0808 07:10:18.743647 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.743659 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.743721 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.743878 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.743879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.743882 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:18.743958 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.743960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.744620 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.744621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.745384 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.745385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.745928 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.745930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.746015 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.746018 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.746091 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.746132 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":95,"threads_started":1}
I20260808 07:10:18.746359 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.743721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:46137 (local address 127.30.54.65:42817)
0808 07:10:18.743817 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.743825 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.743862 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.743990 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.743990 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.743992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:18.744071 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.744072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.744702 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.744703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.745509 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.745511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.746077 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.746078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.746161 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.746164 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.746242 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.746281 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":51,"threads_started":1}
I20260808 07:10:18.746346 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.743811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:42817 (local address 127.30.54.67:46137)
0808 07:10:18.743866 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.743869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.743881 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:10:18.743921 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.743922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.743931 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.743976 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:18.744151 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.744615 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.745597 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.745981 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.746064 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.746072 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.746250 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.746284 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:10:18.746615 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105c1bd357114cf28eee19b47b3d082a" candidate_uuid: "8fd4d830d298407093afc226147ee7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12cce22d691f432bacd9f76aeead2a44" is_pre_election: true
I20260808 07:10:18.746651 1498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105c1bd357114cf28eee19b47b3d082a" candidate_uuid: "8fd4d830d298407093afc226147ee7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" is_pre_election: true
I20260808 07:10:18.746734 1300 raft_consensus.cc:2454] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd4d830d298407093afc226147ee7d4 in term 0.
I20260808 07:10:18.746763 1498 raft_consensus.cc:2454] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd4d830d298407093afc226147ee7d4 in term 0.
I20260808 07:10:18.746959 1004 leader_election.cc:302] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd4d830d298407093afc226147ee7d4, a42cd1f61a6c40bebe01f11ad78e7b4e; no voters:
I20260808 07:10:18.747082 1609 raft_consensus.cc:2781] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:18.747123 1609 raft_consensus.cc:482] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:18.747152 1609 raft_consensus.cc:3037] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.747790 1609 raft_consensus.cc:504] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.747921 1609 leader_election.cc:288] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [CANDIDATE]: Term 1 election: Requested vote from peers 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617), a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137)
I20260808 07:10:18.748107 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105c1bd357114cf28eee19b47b3d082a" candidate_uuid: "8fd4d830d298407093afc226147ee7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12cce22d691f432bacd9f76aeead2a44"
I20260808 07:10:18.748107 1498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105c1bd357114cf28eee19b47b3d082a" candidate_uuid: "8fd4d830d298407093afc226147ee7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e"
I20260808 07:10:18.748180 1498 raft_consensus.cc:3037] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.748181 1300 raft_consensus.cc:3037] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.748937 1498 raft_consensus.cc:2454] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd4d830d298407093afc226147ee7d4 in term 1.
I20260808 07:10:18.748934 1300 raft_consensus.cc:2454] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd4d830d298407093afc226147ee7d4 in term 1.
I20260808 07:10:18.749099 1004 leader_election.cc:302] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12cce22d691f432bacd9f76aeead2a44, 8fd4d830d298407093afc226147ee7d4; no voters:
I20260808 07:10:18.749230 1609 raft_consensus.cc:2781] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:18.749356 1609 raft_consensus.cc:686] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd4d830d298407093afc226147ee7d4, State: Running, Role: LEADER
I20260808 07:10:18.749490 1609 consensus_queue.cc:237] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.749945 1632 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: TxnStatusTablet state changed. Reason: New leader 8fd4d830d298407093afc226147ee7d4. Latest consensus state: current_term: 1 leader_uuid: "8fd4d830d298407093afc226147ee7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.750036 1632 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:10:18.750214 1635 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:10:18.750232 868 catalog_manager.cc:5495] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd4d830d298407093afc226147ee7d4 (127.30.54.65). New cstate: current_term: 1 leader_uuid: "8fd4d830d298407093afc226147ee7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:18.750342 1635 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:10:18.762857 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.760260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.30.54.65:40967)
0808 07:10:18.760306 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.760308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.760316 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:10:18.760395 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.760396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.760413 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.760493 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:18.760616 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.761200 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.762006 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.762209 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.762285 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.762486 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:18.762587 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:18.762676 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.762744 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.762767 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:10:18.762879 1625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.760186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40967 (local address 127.0.0.1:43000)
0808 07:10:18.760255 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.760266 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.760318 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.760502 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.760503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.760504 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:10:18.760552 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.760554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.761279 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.761280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.761941 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.761943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.762287 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.762288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.762354 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.762371 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:10:18.762659 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:10:18.762670 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.762723 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.762776 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:10:18.764576 1498 raft_consensus.cc:1261] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Refusing update from remote peer 8fd4d830d298407093afc226147ee7d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:18.764570 1300 raft_consensus.cc:1261] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Refusing update from remote peer 8fd4d830d298407093afc226147ee7d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:18.764863 1609 consensus_queue.cc:1035] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:18.764964 1632 consensus_queue.cc:1035] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:18.766012 1605 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: TxnStatusTablet state changed. Reason: New leader 8fd4d830d298407093afc226147ee7d4. Latest consensus state: current_term: 1 leader_uuid: "8fd4d830d298407093afc226147ee7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.766108 1605 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:18.766163 1612 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: TxnStatusTablet state changed. Reason: New leader 8fd4d830d298407093afc226147ee7d4. Latest consensus state: current_term: 1 leader_uuid: "8fd4d830d298407093afc226147ee7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.766256 1612 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:18.767151 1609 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fd4d830d298407093afc226147ee7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.767236 1609 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:10:18.767632 1609 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fd4d830d298407093afc226147ee7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.767717 1609 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P 8fd4d830d298407093afc226147ee7d4: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:10:18.767912 1612 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fd4d830d298407093afc226147ee7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.767992 1612 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P 12cce22d691f432bacd9f76aeead2a44: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:18.768302 1605 tablet_replica.cc:439] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fd4d830d298407093afc226147ee7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } } }
I20260808 07:10:18.768380 1605 tablet_replica.cc:442] T 105c1bd357114cf28eee19b47b3d082a P a42cd1f61a6c40bebe01f11ad78e7b4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:10:18.768637 1640 mvcc.cc:205] Tried to move back new op lower bound from 7316164684857450496 to 7316164684798550016. Current Snapshot: MvccSnapshot[applied={T|T < 7316164684857450496}]
I20260808 07:10:18.772152 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.769485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52920 (local address 127.30.54.67:46137)
0808 07:10:18.769691 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.769693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.769700 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:10:18.769728 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.769729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.769744 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.769804 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:18.769907 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.770635 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.771387 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.771687 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.771761 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.771867 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:18.771977 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:18.772045 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.772067 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.772086 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260808 07:10:18.776826 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.774252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40967 (local address 127.30.54.67:57457)
0808 07:10:18.774308 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.774318 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.774363 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.774510 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.774511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.774512 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:10:18.774556 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.774557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.775442 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.775443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.775966 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.775967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.776410 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.776410 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.776458 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.776470 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:10:18.776683 (+ 213us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:10:18.776684 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.776754 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.776782 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 07:10:18.776844 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.774301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:57457 (local address 127.30.54.65:40967)
0808 07:10:18.774344 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.774346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.774355 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:10:18.774431 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.774432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.774441 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.774493 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:18.774627 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.775366 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.776039 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.776334 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.776395 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.776591 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:18.776622 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:18.776678 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.776761 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.776780 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:10:18.779040 1300 raft_consensus.cc:1261] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Refusing update from remote peer a42cd1f61a6c40bebe01f11ad78e7b4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:18.779155 1103 raft_consensus.cc:1261] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Refusing update from remote peer a42cd1f61a6c40bebe01f11ad78e7b4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:18.779289 1614 consensus_queue.cc:1035] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:18.779384 1605 consensus_queue.cc:1035] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:18.780556 1639 mvcc.cc:205] Tried to move back new op lower bound from 7316164684917211136 to 7316164684294127616. Current Snapshot: MvccSnapshot[applied={T|T < 7316164684917211136}]
I20260808 07:10:18.781019 1638 mvcc.cc:205] Tried to move back new op lower bound from 7316164684917211136 to 7316164684294127616. Current Snapshot: MvccSnapshot[applied={T|T < 7316164684917211136}]
I20260808 07:10:18.787788 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.785151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:35027 (local address 127.30.54.67:46137)
0808 07:10:18.785183 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.785185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.785193 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:10:18.785265 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.785266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.785274 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.785329 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:18.785419 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.786112 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.787081 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.787313 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.787406 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.787573 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:18.787604 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:18.787650 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.787707 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.787724 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:10:18.787906 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.784978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:46137 (local address 127.30.54.65:35027)
0808 07:10:18.785145 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.785156 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.785204 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.785331 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.785331 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.785332 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:10:18.785376 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.785377 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.786184 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.786185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.787007 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.787010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.787410 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.787411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.787475 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.787495 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:10:18.787650 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:10:18.787652 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.787703 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.787741 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260808 07:10:18.796900 868 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:18.796993 868 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:18.797714 868 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69: Sending DeleteTablet for 3 replicas of tablet 69081290170b4d5689b19cf899223345
I20260808 07:10:18.798040 1276 tablet_service.cc:1514] Processing DeleteTablet for tablet 69081290170b4d5689b19cf899223345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:18 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:58640
I20260808 07:10:18.798082 1071 tablet_service.cc:1514] Processing DeleteTablet for tablet 69081290170b4d5689b19cf899223345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:18 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:42994
I20260808 07:10:18.798264 1470 tablet_service.cc:1514] Processing DeleteTablet for tablet 69081290170b4d5689b19cf899223345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:18 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:52912
I20260808 07:10:18.798522 1665 tablet_replica.cc:330] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: stopping tablet replica
I20260808 07:10:18.798642 1665 raft_consensus.cc:2229] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:18.798794 1665 raft_consensus.cc:2258] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:18.798969 868 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:18.799111 868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:18.799525 1665 ts_tablet_manager.cc:1905] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:18.800226 1276 tablet_service.cc:1467] Processing CreateTablet for tablet 082c0a0f002948c8a51b279df1f04659 (DEFAULT_TABLE table=test-table [id=05cfb9978bf84e94a3c477cf2f19b7a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:18.800369 1276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 082c0a0f002948c8a51b279df1f04659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.801592 1665 ts_tablet_manager.cc:1918] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:10:18.801733 1665 log.cc:1196] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/69081290170b4d5689b19cf899223345
I20260808 07:10:18.801846 1603 tablet_bootstrap.cc:492] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: Bootstrap starting.
I20260808 07:10:18.802174 1665 ts_tablet_manager.cc:1939] T 69081290170b4d5689b19cf899223345 P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting consensus metadata
I20260808 07:10:18.802233 1603 tablet_bootstrap.cc:654] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.802776 1470 tablet_service.cc:1467] Processing CreateTablet for tablet 082c0a0f002948c8a51b279df1f04659 (DEFAULT_TABLE table=test-table [id=05cfb9978bf84e94a3c477cf2f19b7a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:18.802806 852 catalog_manager.cc:4841] TS a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137): tablet 69081290170b4d5689b19cf899223345 (table test-table [id=605cf118923e4bd58ad3449f54c96e82]) successfully deleted
I20260808 07:10:18.802922 1470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 082c0a0f002948c8a51b279df1f04659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.803076 1071 tablet_service.cc:1467] Processing CreateTablet for tablet 082c0a0f002948c8a51b279df1f04659 (DEFAULT_TABLE table=test-table [id=05cfb9978bf84e94a3c477cf2f19b7a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:18.803169 1071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 082c0a0f002948c8a51b279df1f04659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:18.804179 1596 tablet_bootstrap.cc:492] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: Bootstrap starting.
I20260808 07:10:18.804569 1596 tablet_bootstrap.cc:654] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.804577 1599 tablet_bootstrap.cc:492] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: Bootstrap starting.
I20260808 07:10:18.804998 1599 tablet_bootstrap.cc:654] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:18.805464 1596 tablet_bootstrap.cc:492] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: No bootstrap required, opened a new log
I20260808 07:10:18.805513 1596 ts_tablet_manager.cc:1397] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:10:18.805655 1596 raft_consensus.cc:348] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.805729 1596 raft_consensus.cc:374] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.805756 1596 raft_consensus.cc:729] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42cd1f61a6c40bebe01f11ad78e7b4e, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.805801 1596 consensus_queue.cc:260] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.805918 1596 ts_tablet_manager.cc:1428] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.806107 1603 tablet_bootstrap.cc:492] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: No bootstrap required, opened a new log
I20260808 07:10:18.806152 1603 ts_tablet_manager.cc:1397] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:10:18.806280 1603 raft_consensus.cc:348] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.806388 1603 raft_consensus.cc:374] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.806413 1603 raft_consensus.cc:729] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12cce22d691f432bacd9f76aeead2a44, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.806455 1603 consensus_queue.cc:260] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.806567 1603 ts_tablet_manager.cc:1428] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:18.807128 1599 tablet_bootstrap.cc:492] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: No bootstrap required, opened a new log
I20260808 07:10:18.807176 1599 ts_tablet_manager.cc:1397] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:10:18.807336 1599 raft_consensus.cc:348] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.807421 1599 raft_consensus.cc:374] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:18.807449 1599 raft_consensus.cc:729] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd4d830d298407093afc226147ee7d4, State: Initialized, Role: FOLLOWER
I20260808 07:10:18.807490 1599 consensus_queue.cc:260] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.807577 1599 ts_tablet_manager.cc:1428] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:18.808115 1664 tablet_replica.cc:330] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: stopping tablet replica
I20260808 07:10:18.808254 1664 raft_consensus.cc:2229] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:18.808307 1664 raft_consensus.cc:2258] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:18.808488 1663 tablet_replica.cc:330] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: stopping tablet replica
I20260808 07:10:18.808595 1663 raft_consensus.cc:2229] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:18.808658 1663 raft_consensus.cc:2258] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:18.808830 1664 ts_tablet_manager.cc:1905] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:18.809051 1663 ts_tablet_manager.cc:1905] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:18.810838 1664 ts_tablet_manager.cc:1918] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:10:18.810981 1664 log.cc:1196] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/69081290170b4d5689b19cf899223345
I20260808 07:10:18.811034 1663 ts_tablet_manager.cc:1918] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:10:18.811102 1663 log.cc:1196] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/69081290170b4d5689b19cf899223345
I20260808 07:10:18.811386 1663 ts_tablet_manager.cc:1939] T 69081290170b4d5689b19cf899223345 P 12cce22d691f432bacd9f76aeead2a44: Deleting consensus metadata
I20260808 07:10:18.811399 1664 ts_tablet_manager.cc:1939] T 69081290170b4d5689b19cf899223345 P 8fd4d830d298407093afc226147ee7d4: Deleting consensus metadata
I20260808 07:10:18.811780 852 catalog_manager.cc:4841] TS 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617): tablet 69081290170b4d5689b19cf899223345 (table test-table [id=605cf118923e4bd58ad3449f54c96e82]) successfully deleted
I20260808 07:10:18.811937 853 catalog_manager.cc:4841] TS 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967): tablet 69081290170b4d5689b19cf899223345 (table test-table [id=605cf118923e4bd58ad3449f54c96e82]) successfully deleted
I20260808 07:10:18.990314 1612 raft_consensus.cc:482] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:18.990418 1612 raft_consensus.cc:504] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.990798 1612 leader_election.cc:288] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967), a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137)
I20260808 07:10:18.994714 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.991142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:44169 (local address 127.30.54.67:46137)
0808 07:10:18.991196 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.991198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.991208 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:18.991241 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.991242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.991254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.991319 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:18.991537 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.992289 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.993796 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.994342 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.994423 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.994432 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.994603 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.994641 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:10:18.994714 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.991050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:52265 (local address 127.30.54.65:40967)
0808 07:10:18.991129 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:18.991131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:18.991191 (+ 60us) server_negotiation.cc:407] Connection header received
0808 07:10:18.991245 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:18.991246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:18.991258 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:18.991319 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:18.991536 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.992289 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.993795 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.994342 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.994427 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.994434 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:18.994603 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:10:18.994641 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:10:18.994747 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.991008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:46137 (local address 127.30.54.66:44169)
0808 07:10:18.991117 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.991129 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.991187 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.991337 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.991338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.991341 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:18.991447 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.991451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.992373 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.992374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.993701 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.993704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.994432 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.994432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.994516 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.994523 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.994582 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.994641 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":97,"thread_start_us":59,"threads_started":1}
I20260808 07:10:18.994755 1671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.990982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40967 (local address 127.30.54.66:52265)
0808 07:10:18.991116 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.991129 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:18.991189 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:18.991338 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:18.991338 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:18.991342 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:18.991447 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.991450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.992374 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.992375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.993701 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:18.993704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:18.994430 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:18.994432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:18.994516 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:18.994523 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:10:18.994582 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:10:18.994641 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":38,"thread_start_us":66,"threads_started":1}
I20260808 07:10:18.995019 1498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "082c0a0f002948c8a51b279df1f04659" candidate_uuid: "12cce22d691f432bacd9f76aeead2a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" is_pre_election: true
I20260808 07:10:18.995133 1498 raft_consensus.cc:2454] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12cce22d691f432bacd9f76aeead2a44 in term 0.
I20260808 07:10:18.995159 1103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "082c0a0f002948c8a51b279df1f04659" candidate_uuid: "12cce22d691f432bacd9f76aeead2a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd4d830d298407093afc226147ee7d4" is_pre_election: true
I20260808 07:10:18.995246 1103 raft_consensus.cc:2454] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12cce22d691f432bacd9f76aeead2a44 in term 0.
I20260808 07:10:18.995301 1199 leader_election.cc:302] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12cce22d691f432bacd9f76aeead2a44, a42cd1f61a6c40bebe01f11ad78e7b4e; no voters:
I20260808 07:10:18.995481 1612 raft_consensus.cc:2781] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:18.995524 1612 raft_consensus.cc:482] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:18.995539 1612 raft_consensus.cc:3037] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.996244 1612 raft_consensus.cc:504] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.996385 1612 leader_election.cc:288] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967), a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137)
I20260808 07:10:18.996538 1498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "082c0a0f002948c8a51b279df1f04659" candidate_uuid: "12cce22d691f432bacd9f76aeead2a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e"
I20260808 07:10:18.996537 1103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "082c0a0f002948c8a51b279df1f04659" candidate_uuid: "12cce22d691f432bacd9f76aeead2a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd4d830d298407093afc226147ee7d4"
I20260808 07:10:18.996609 1498 raft_consensus.cc:3037] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.996609 1103 raft_consensus.cc:3037] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:18.997356 1498 raft_consensus.cc:2454] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12cce22d691f432bacd9f76aeead2a44 in term 1.
I20260808 07:10:18.997356 1103 raft_consensus.cc:2454] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12cce22d691f432bacd9f76aeead2a44 in term 1.
I20260808 07:10:18.997537 1200 leader_election.cc:302] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12cce22d691f432bacd9f76aeead2a44, 8fd4d830d298407093afc226147ee7d4; no voters:
I20260808 07:10:18.997637 1612 raft_consensus.cc:2781] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:18.997767 1612 raft_consensus.cc:686] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 1 LEADER]: Becoming Leader. State: Replica: 12cce22d691f432bacd9f76aeead2a44, State: Running, Role: LEADER
I20260808 07:10:18.997850 1612 consensus_queue.cc:237] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:18.998558 868 catalog_manager.cc:5495] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66). New cstate: current_term: 1 leader_uuid: "12cce22d691f432bacd9f76aeead2a44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:19.037314 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:19.034897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58656 (local address 127.30.54.66:46617)
0808 07:10:19.034938 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:19.034941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:19.034949 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:10:19.035010 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:19.035011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:19.035032 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:19.035126 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:19.035262 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.035748 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.036487 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.036697 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.036775 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:19.036953 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:19.037067 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:19.037148 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:19.037201 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 07:10:19.037221 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:10:19.042238 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:19.039843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:56813 (local address 127.30.54.65:40967)
0808 07:10:19.039913 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:19.039915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:19.040026 (+ 111us) server_negotiation.cc:407] Connection header received
0808 07:10:19.040094 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:19.040095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:19.040107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:19.040172 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:19.040295 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.040760 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.041518 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.041725 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.041813 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:19.041986 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:19.042020 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:19.042088 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:19.042151 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:10:19.042169 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:10:19.042487 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:19.039745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40967 (local address 127.30.54.66:56813)
0808 07:10:19.039950 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:19.039962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:19.040042 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:19.040160 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:19.040161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:19.040164 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:10:19.040222 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:19.040223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.040855 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.040856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:19.041428 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:19.041430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.041821 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.041822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:19.041873 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:19.041892 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:10:19.042091 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:10:19.042094 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:19.042146 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:10:19.042186 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
I20260808 07:10:19.044587 1498 raft_consensus.cc:1261] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Refusing update from remote peer 12cce22d691f432bacd9f76aeead2a44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:19.044586 1103 raft_consensus.cc:1261] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Refusing update from remote peer 12cce22d691f432bacd9f76aeead2a44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:19.044917 1674 consensus_queue.cc:1035] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:19.045032 1612 consensus_queue.cc:1035] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:19.052433 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:19.049947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46617 (local address 127.30.54.65:34613)
0808 07:10:19.049991 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:19.050000 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:19.050040 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:19.050162 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:19.050163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:19.050164 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:10:19.050206 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:19.050207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.051013 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.051014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:19.051549 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:19.051551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.052008 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.052009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:19.052068 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:19.052080 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:10:19.052271 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:10:19.052274 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:19.052314 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 07:10:19.052369 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 07:10:19.052627 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:19.049985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:34613 (local address 127.30.54.66:46617)
0808 07:10:19.050023 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:19.050025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:19.050033 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:10:19.050085 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:19.050086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:19.050092 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:19.050137 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:19.050278 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.050934 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.051615 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:19.051839 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:19.051913 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:19.052153 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:19.052184 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:19.052242 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:19.052531 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 07:10:19.052554 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:10:19.052958 1640 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:10:20.574903 869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:20.575033 869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:20.575827 869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69: Sending DeleteTablet for 3 replicas of tablet 082c0a0f002948c8a51b279df1f04659
I20260808 07:10:20.576114 1470 tablet_service.cc:1514] Processing DeleteTablet for tablet 082c0a0f002948c8a51b279df1f04659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:20 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:52912
I20260808 07:10:20.576140 1276 tablet_service.cc:1514] Processing DeleteTablet for tablet 082c0a0f002948c8a51b279df1f04659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:20 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:58640
I20260808 07:10:20.576341 1721 tablet_replica.cc:330] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: stopping tablet replica
I20260808 07:10:20.576407 1721 raft_consensus.cc:2229] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:20.576429 1071 tablet_service.cc:1514] Processing DeleteTablet for tablet 082c0a0f002948c8a51b279df1f04659 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:20 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:42994
I20260808 07:10:20.576511 1721 raft_consensus.cc:2258] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:20.576730 1721 ts_tablet_manager.cc:1905] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:20.578681 1721 ts_tablet_manager.cc:1918] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:10:20.578732 1721 log.cc:1196] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/082c0a0f002948c8a51b279df1f04659
I20260808 07:10:20.578989 1722 tablet_replica.cc:330] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: stopping tablet replica
I20260808 07:10:20.579008 1721 ts_tablet_manager.cc:1939] T 082c0a0f002948c8a51b279df1f04659 P 12cce22d691f432bacd9f76aeead2a44: Deleting consensus metadata
I20260808 07:10:20.579101 1720 tablet_replica.cc:330] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: stopping tablet replica
I20260808 07:10:20.579165 1720 raft_consensus.cc:2229] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:20.579216 1720 raft_consensus.cc:2258] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:20.579298 1722 raft_consensus.cc:2229] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:20.579352 1722 raft_consensus.cc:2258] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:20.579427 852 catalog_manager.cc:4841] TS 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617): tablet 082c0a0f002948c8a51b279df1f04659 (table test-table [id=05cfb9978bf84e94a3c477cf2f19b7a9]) successfully deleted
I20260808 07:10:20.579707 1722 ts_tablet_manager.cc:1905] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:20.580935 1720 ts_tablet_manager.cc:1905] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:20.581378 1722 ts_tablet_manager.cc:1918] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:10:20.581454 1722 log.cc:1196] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/082c0a0f002948c8a51b279df1f04659
I20260808 07:10:20.581666 1722 ts_tablet_manager.cc:1939] T 082c0a0f002948c8a51b279df1f04659 P 8fd4d830d298407093afc226147ee7d4: Deleting consensus metadata
I20260808 07:10:20.582139 853 catalog_manager.cc:4841] TS 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967): tablet 082c0a0f002948c8a51b279df1f04659 (table test-table [id=05cfb9978bf84e94a3c477cf2f19b7a9]) successfully deleted
I20260808 07:10:20.582649 1720 ts_tablet_manager.cc:1918] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:10:20.582703 1720 log.cc:1196] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/082c0a0f002948c8a51b279df1f04659
I20260808 07:10:20.582959 1720 ts_tablet_manager.cc:1939] T 082c0a0f002948c8a51b279df1f04659 P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting consensus metadata
I20260808 07:10:20.583405 852 catalog_manager.cc:4841] TS a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137): tablet 082c0a0f002948c8a51b279df1f04659 (table test-table [id=05cfb9978bf84e94a3c477cf2f19b7a9]) successfully deleted
Aug 08 07:10:20 dist-test-slave-znh6 krb5kdc[808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173020, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:10:20 dist-test-slave-znh6 krb5kdc[808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173020, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:20.606566 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:20.594126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.30.54.126:39041)
0808 07:10:20.594326 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:20.594329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:20.594422 (+ 93us) server_negotiation.cc:407] Connection header received
0808 07:10:20.594508 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:20.594509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:20.594523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:20.594608 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:20.595276 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:20.595803 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:20.596633 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:20.596842 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:20.596933 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:20.604002 (+ 7069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:20.604003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:20.604003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:20.604010 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:20.605535 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:20.605834 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:20.605834 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:20.605835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:20.605897 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:20.606124 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:20.606124 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:20.606124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:20.606226 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:20.606300 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:20.606418 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 07:10:20.606489 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
I20260808 07:10:20.608088 869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:20.608263 869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:20.609743 1071 tablet_service.cc:1467] Processing CreateTablet for tablet ce0ce89e2ff94bb3a92743b240362abd (DEFAULT_TABLE table=test-table [id=72a85e2f7b4941bcb48cc002b2f712c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:20.609773 1276 tablet_service.cc:1467] Processing CreateTablet for tablet ce0ce89e2ff94bb3a92743b240362abd (DEFAULT_TABLE table=test-table [id=72a85e2f7b4941bcb48cc002b2f712c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:20.609771 1470 tablet_service.cc:1467] Processing CreateTablet for tablet ce0ce89e2ff94bb3a92743b240362abd (DEFAULT_TABLE table=test-table [id=72a85e2f7b4941bcb48cc002b2f712c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:20.609901 1276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce0ce89e2ff94bb3a92743b240362abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:20.609901 1071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce0ce89e2ff94bb3a92743b240362abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:20.609927 1470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce0ce89e2ff94bb3a92743b240362abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:20.611145 1735 tablet_bootstrap.cc:492] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: Bootstrap starting.
I20260808 07:10:20.611490 1737 tablet_bootstrap.cc:492] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: Bootstrap starting.
I20260808 07:10:20.611635 1735 tablet_bootstrap.cc:654] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:20.611914 1737 tablet_bootstrap.cc:654] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:20.612255 1738 tablet_bootstrap.cc:492] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: Bootstrap starting.
I20260808 07:10:20.612293 1735 tablet_bootstrap.cc:492] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: No bootstrap required, opened a new log
I20260808 07:10:20.612341 1735 ts_tablet_manager.cc:1397] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:20.612465 1735 raft_consensus.cc:348] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.612538 1735 raft_consensus.cc:374] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:20.612555 1735 raft_consensus.cc:729] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42cd1f61a6c40bebe01f11ad78e7b4e, State: Initialized, Role: FOLLOWER
I20260808 07:10:20.612612 1735 consensus_queue.cc:260] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.612650 1738 tablet_bootstrap.cc:654] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:20.612793 1735 ts_tablet_manager.cc:1428] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:20.613097 1737 tablet_bootstrap.cc:492] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: No bootstrap required, opened a new log
I20260808 07:10:20.613157 1737 ts_tablet_manager.cc:1397] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:20.613304 1737 raft_consensus.cc:348] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.613368 1738 tablet_bootstrap.cc:492] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: No bootstrap required, opened a new log
I20260808 07:10:20.613392 1737 raft_consensus.cc:374] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:20.613415 1737 raft_consensus.cc:729] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd4d830d298407093afc226147ee7d4, State: Initialized, Role: FOLLOWER
I20260808 07:10:20.613422 1738 ts_tablet_manager.cc:1397] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:20.613481 1737 consensus_queue.cc:260] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.613540 1738 raft_consensus.cc:348] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.613621 1738 raft_consensus.cc:374] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:20.613651 1738 raft_consensus.cc:729] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12cce22d691f432bacd9f76aeead2a44, State: Initialized, Role: FOLLOWER
I20260808 07:10:20.613692 1737 ts_tablet_manager.cc:1428] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:20.613718 1738 consensus_queue.cc:260] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.613885 1738 ts_tablet_manager.cc:1428] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:20.775352 1741 raft_consensus.cc:482] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:20.775455 1741 raft_consensus.cc:504] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.775640 1741 leader_election.cc:288] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967), 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617)
I20260808 07:10:20.775890 1103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce0ce89e2ff94bb3a92743b240362abd" candidate_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd4d830d298407093afc226147ee7d4" is_pre_election: true
I20260808 07:10:20.775890 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce0ce89e2ff94bb3a92743b240362abd" candidate_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12cce22d691f432bacd9f76aeead2a44" is_pre_election: true
I20260808 07:10:20.776010 1300 raft_consensus.cc:2454] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a42cd1f61a6c40bebe01f11ad78e7b4e in term 0.
I20260808 07:10:20.776010 1103 raft_consensus.cc:2454] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a42cd1f61a6c40bebe01f11ad78e7b4e in term 0.
I20260808 07:10:20.776185 1404 leader_election.cc:302] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd4d830d298407093afc226147ee7d4, a42cd1f61a6c40bebe01f11ad78e7b4e; no voters:
I20260808 07:10:20.776299 1741 raft_consensus.cc:2781] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:20.776381 1741 raft_consensus.cc:482] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:20.776397 1741 raft_consensus.cc:3037] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:20.777088 1741 raft_consensus.cc:504] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.777236 1741 leader_election.cc:288] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [CANDIDATE]: Term 1 election: Requested vote from peers 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967), 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617)
I20260808 07:10:20.777388 1103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce0ce89e2ff94bb3a92743b240362abd" candidate_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd4d830d298407093afc226147ee7d4"
I20260808 07:10:20.777420 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce0ce89e2ff94bb3a92743b240362abd" candidate_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12cce22d691f432bacd9f76aeead2a44"
I20260808 07:10:20.777478 1103 raft_consensus.cc:3037] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:20.777487 1300 raft_consensus.cc:3037] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:20.778245 1103 raft_consensus.cc:2454] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a42cd1f61a6c40bebe01f11ad78e7b4e in term 1.
I20260808 07:10:20.778244 1300 raft_consensus.cc:2454] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a42cd1f61a6c40bebe01f11ad78e7b4e in term 1.
I20260808 07:10:20.778461 1404 leader_election.cc:302] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd4d830d298407093afc226147ee7d4, a42cd1f61a6c40bebe01f11ad78e7b4e; no voters:
I20260808 07:10:20.778568 1741 raft_consensus.cc:2781] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:20.778752 1741 raft_consensus.cc:686] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 LEADER]: Becoming Leader. State: Replica: a42cd1f61a6c40bebe01f11ad78e7b4e, State: Running, Role: LEADER
I20260808 07:10:20.778856 1741 consensus_queue.cc:237] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.779495 868 catalog_manager.cc:5495] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e reported cstate change: term changed from 0 to 1, leader changed from <none> to a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67). New cstate: current_term: 1 leader_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } health_report { overall_health: HEALTHY } } }
I20260808 07:10:20.796726 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:20.794268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52928 (local address 127.30.54.67:46137)
0808 07:10:20.794460 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:20.794462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:20.794469 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:10:20.794488 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:20.794488 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:20.794499 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:20.794560 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:20.794671 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:20.795230 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:20.796058 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:20.796251 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:20.796306 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:20.796474 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:20.796503 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:20.796544 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:20.796607 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:10:20.796626 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":131,"threads_started":1}
I20260808 07:10:20.799062 1103 raft_consensus.cc:1261] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Refusing update from remote peer a42cd1f61a6c40bebe01f11ad78e7b4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:20.799124 1300 raft_consensus.cc:1261] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Refusing update from remote peer a42cd1f61a6c40bebe01f11ad78e7b4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:20.799288 1741 consensus_queue.cc:1035] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:20.799383 1743 consensus_queue.cc:1035] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:20.808929 868 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:20.809047 868 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:20.809688 868 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69: Sending DeleteTablet for 3 replicas of tablet ce0ce89e2ff94bb3a92743b240362abd
I20260808 07:10:20.809978 1276 tablet_service.cc:1514] Processing DeleteTablet for tablet ce0ce89e2ff94bb3a92743b240362abd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:20 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:58640
I20260808 07:10:20.809975 1071 tablet_service.cc:1514] Processing DeleteTablet for tablet ce0ce89e2ff94bb3a92743b240362abd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:20 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:42994
I20260808 07:10:20.810072 1722 tablet_replica.cc:330] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: stopping tablet replica
I20260808 07:10:20.810079 1721 tablet_replica.cc:330] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: stopping tablet replica
I20260808 07:10:20.810071 1470 tablet_service.cc:1514] Processing DeleteTablet for tablet ce0ce89e2ff94bb3a92743b240362abd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:20 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:52912
I20260808 07:10:20.810124 1721 raft_consensus.cc:2229] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:20.810125 1722 raft_consensus.cc:2229] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:20.810158 1721 raft_consensus.cc:2258] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:20.810161 1722 raft_consensus.cc:2258] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:20.810228 1720 tablet_replica.cc:330] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: stopping tablet replica
I20260808 07:10:20.810285 1720 raft_consensus.cc:2229] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:20.810338 1720 raft_consensus.cc:2258] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:20.810462 1721 ts_tablet_manager.cc:1905] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:20.810529 1722 ts_tablet_manager.cc:1905] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:20.810498 868 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:20.810634 868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:20.810920 1720 ts_tablet_manager.cc:1905] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:20.811795 1470 tablet_service.cc:1467] Processing CreateTablet for tablet e95d564d8d1d4fa7881528f53270364e (DEFAULT_TABLE table=test-table [id=43e8bf6915ed4f05a5274ca2cf2bcc75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:20.811901 1470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e95d564d8d1d4fa7881528f53270364e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:20.812393 1722 ts_tablet_manager.cc:1918] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:10:20.812402 1721 ts_tablet_manager.cc:1918] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:10:20.812454 1721 log.cc:1196] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/ce0ce89e2ff94bb3a92743b240362abd
I20260808 07:10:20.812453 1722 log.cc:1196] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/ce0ce89e2ff94bb3a92743b240362abd
I20260808 07:10:20.812539 1720 ts_tablet_manager.cc:1918] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:10:20.812592 1720 log.cc:1196] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/ce0ce89e2ff94bb3a92743b240362abd
I20260808 07:10:20.812693 1721 ts_tablet_manager.cc:1939] T ce0ce89e2ff94bb3a92743b240362abd P 12cce22d691f432bacd9f76aeead2a44: Deleting consensus metadata
I20260808 07:10:20.812700 1722 ts_tablet_manager.cc:1939] T ce0ce89e2ff94bb3a92743b240362abd P 8fd4d830d298407093afc226147ee7d4: Deleting consensus metadata
I20260808 07:10:20.812804 1720 ts_tablet_manager.cc:1939] T ce0ce89e2ff94bb3a92743b240362abd P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting consensus metadata
I20260808 07:10:20.813124 852 catalog_manager.cc:4841] TS a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137): tablet ce0ce89e2ff94bb3a92743b240362abd (table test-table [id=72a85e2f7b4941bcb48cc002b2f712c4]) successfully deleted
I20260808 07:10:20.813268 1735 tablet_bootstrap.cc:492] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: Bootstrap starting.
I20260808 07:10:20.813405 853 catalog_manager.cc:4841] TS 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967): tablet ce0ce89e2ff94bb3a92743b240362abd (table test-table [id=72a85e2f7b4941bcb48cc002b2f712c4]) successfully deleted
I20260808 07:10:20.813426 1071 tablet_service.cc:1467] Processing CreateTablet for tablet e95d564d8d1d4fa7881528f53270364e (DEFAULT_TABLE table=test-table [id=43e8bf6915ed4f05a5274ca2cf2bcc75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:20.813532 1071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e95d564d8d1d4fa7881528f53270364e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:20.813639 1735 tablet_bootstrap.cc:654] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:20.813812 852 catalog_manager.cc:4841] TS 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617): tablet ce0ce89e2ff94bb3a92743b240362abd (table test-table [id=72a85e2f7b4941bcb48cc002b2f712c4]) successfully deleted
I20260808 07:10:20.813889 1276 tablet_service.cc:1467] Processing CreateTablet for tablet e95d564d8d1d4fa7881528f53270364e (DEFAULT_TABLE table=test-table [id=43e8bf6915ed4f05a5274ca2cf2bcc75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:20.813990 1276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e95d564d8d1d4fa7881528f53270364e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:20.814584 1737 tablet_bootstrap.cc:492] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4: Bootstrap starting.
I20260808 07:10:20.814903 1738 tablet_bootstrap.cc:492] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: Bootstrap starting.
I20260808 07:10:20.814952 1737 tablet_bootstrap.cc:654] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:20.815220 1738 tablet_bootstrap.cc:654] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:20.815358 1735 tablet_bootstrap.cc:492] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: No bootstrap required, opened a new log
I20260808 07:10:20.815403 1735 ts_tablet_manager.cc:1397] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:20.815547 1737 tablet_bootstrap.cc:492] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4: No bootstrap required, opened a new log
I20260808 07:10:20.815531 1735 raft_consensus.cc:348] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.815598 1737 ts_tablet_manager.cc:1397] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:20.815616 1735 raft_consensus.cc:374] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:20.815644 1735 raft_consensus.cc:729] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42cd1f61a6c40bebe01f11ad78e7b4e, State: Initialized, Role: FOLLOWER
I20260808 07:10:20.815688 1735 consensus_queue.cc:260] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.815699 1737 raft_consensus.cc:348] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.815769 1737 raft_consensus.cc:374] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:20.815779 1735 ts_tablet_manager.cc:1428] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:20.815781 1738 tablet_bootstrap.cc:492] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: No bootstrap required, opened a new log
I20260808 07:10:20.815791 1737 raft_consensus.cc:729] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd4d830d298407093afc226147ee7d4, State: Initialized, Role: FOLLOWER
I20260808 07:10:20.815824 1738 ts_tablet_manager.cc:1397] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:20.815836 1737 consensus_queue.cc:260] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.815934 1737 ts_tablet_manager.cc:1428] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:20.815963 1738 raft_consensus.cc:348] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.816026 1738 raft_consensus.cc:374] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:20.816046 1738 raft_consensus.cc:729] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12cce22d691f432bacd9f76aeead2a44, State: Initialized, Role: FOLLOWER
I20260808 07:10:20.816090 1738 consensus_queue.cc:260] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.816183 1738 ts_tablet_manager.cc:1428] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:20.863216 1689 raft_consensus.cc:482] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:20.863343 1689 raft_consensus.cc:504] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.863559 1689 leader_election.cc:288] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967), a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137)
I20260808 07:10:20.863782 1103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e95d564d8d1d4fa7881528f53270364e" candidate_uuid: "12cce22d691f432bacd9f76aeead2a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd4d830d298407093afc226147ee7d4" is_pre_election: true
I20260808 07:10:20.863762 1498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e95d564d8d1d4fa7881528f53270364e" candidate_uuid: "12cce22d691f432bacd9f76aeead2a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" is_pre_election: true
I20260808 07:10:20.863896 1103 raft_consensus.cc:2454] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12cce22d691f432bacd9f76aeead2a44 in term 0.
I20260808 07:10:20.863893 1498 raft_consensus.cc:2454] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12cce22d691f432bacd9f76aeead2a44 in term 0.
I20260808 07:10:20.864065 1200 leader_election.cc:302] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12cce22d691f432bacd9f76aeead2a44, 8fd4d830d298407093afc226147ee7d4; no voters:
I20260808 07:10:20.864188 1689 raft_consensus.cc:2781] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:20.864239 1689 raft_consensus.cc:482] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:20.864269 1689 raft_consensus.cc:3037] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:20.865017 1689 raft_consensus.cc:504] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.865162 1689 leader_election.cc:288] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967), a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137)
I20260808 07:10:20.865319 1103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e95d564d8d1d4fa7881528f53270364e" candidate_uuid: "12cce22d691f432bacd9f76aeead2a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd4d830d298407093afc226147ee7d4"
I20260808 07:10:20.865314 1498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e95d564d8d1d4fa7881528f53270364e" candidate_uuid: "12cce22d691f432bacd9f76aeead2a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e"
I20260808 07:10:20.865414 1103 raft_consensus.cc:3037] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:20.865414 1498 raft_consensus.cc:3037] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:20.866021 1103 raft_consensus.cc:2454] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12cce22d691f432bacd9f76aeead2a44 in term 1.
I20260808 07:10:20.866184 1498 raft_consensus.cc:2454] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12cce22d691f432bacd9f76aeead2a44 in term 1.
I20260808 07:10:20.866228 1200 leader_election.cc:302] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12cce22d691f432bacd9f76aeead2a44, 8fd4d830d298407093afc226147ee7d4; no voters:
I20260808 07:10:20.866374 1689 raft_consensus.cc:2781] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:20.866441 1689 raft_consensus.cc:686] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 1 LEADER]: Becoming Leader. State: Replica: 12cce22d691f432bacd9f76aeead2a44, State: Running, Role: LEADER
I20260808 07:10:20.866492 1689 consensus_queue.cc:237] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } }
I20260808 07:10:20.867063 868 catalog_manager.cc:5495] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66). New cstate: current_term: 1 leader_uuid: "12cce22d691f432bacd9f76aeead2a44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12cce22d691f432bacd9f76aeead2a44" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:20.878810 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:20.876523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.30.54.66:46617)
0808 07:10:20.876673 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:20.876675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:20.876682 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:10:20.876705 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:20.876706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:20.876715 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:20.876774 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:20.876870 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:20.877376 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:20.878151 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:20.878365 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:20.878418 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:20.878598 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:20.878626 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:20.878665 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:20.878696 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 07:10:20.878713 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260808 07:10:20.881274 1498 raft_consensus.cc:1261] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Refusing update from remote peer 12cce22d691f432bacd9f76aeead2a44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:20.881280 1103 raft_consensus.cc:1261] T e95d564d8d1d4fa7881528f53270364e P 8fd4d830d298407093afc226147ee7d4 [term 1 FOLLOWER]: Refusing update from remote peer 12cce22d691f432bacd9f76aeead2a44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:20.881506 1710 consensus_queue.cc:1035] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd4d830d298407093afc226147ee7d4" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:20.881628 1689 consensus_queue.cc:1035] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a42cd1f61a6c40bebe01f11ad78e7b4e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:10:20.885921 1703 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:10:22.408427 869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:22.408540 869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:22.409425 869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12bbe7740c944c26a0cc62ef86ed8e69: Sending DeleteTablet for 3 replicas of tablet e95d564d8d1d4fa7881528f53270364e
I20260808 07:10:22.409685 1066 tablet_service.cc:1514] Processing DeleteTablet for tablet e95d564d8d1d4fa7881528f53270364e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:22 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:42994
I20260808 07:10:22.409693 1276 tablet_service.cc:1514] Processing DeleteTablet for tablet e95d564d8d1d4fa7881528f53270364e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:22 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:58640
I20260808 07:10:22.409788 1470 tablet_service.cc:1514] Processing DeleteTablet for tablet e95d564d8d1d4fa7881528f53270364e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:22 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:52912
I20260808 07:10:22.409912 1805 tablet_replica.cc:330] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: stopping tablet replica
I20260808 07:10:22.409950 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 973
I20260808 07:10:22.409973 1805 raft_consensus.cc:2229] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:22.412231 1805 raft_consensus.cc:2258] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:22.413793 1805 ts_tablet_manager.cc:1905] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:22.415602 1805 ts_tablet_manager.cc:1918] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:10:22.415654 1805 log.cc:1196] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/e95d564d8d1d4fa7881528f53270364e
I20260808 07:10:22.415876 1805 ts_tablet_manager.cc:1939] T e95d564d8d1d4fa7881528f53270364e P 12cce22d691f432bacd9f76aeead2a44: Deleting consensus metadata
I20260808 07:10:22.416039 1808 tablet_replica.cc:330] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: stopping tablet replica
I20260808 07:10:22.416256 1808 raft_consensus.cc:2229] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:22.416316 1808 raft_consensus.cc:2258] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:22.416512 852 catalog_manager.cc:4841] TS 12cce22d691f432bacd9f76aeead2a44 (127.30.54.66:46617): tablet e95d564d8d1d4fa7881528f53270364e (table test-table [id=43e8bf6915ed4f05a5274ca2cf2bcc75]) successfully deleted
I20260808 07:10:22.416597 1808 ts_tablet_manager.cc:1905] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:22.417932 1808 ts_tablet_manager.cc:1918] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:10:22.417984 1808 log.cc:1196] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/e95d564d8d1d4fa7881528f53270364e
I20260808 07:10:22.418221 1808 ts_tablet_manager.cc:1939] T e95d564d8d1d4fa7881528f53270364e P a42cd1f61a6c40bebe01f11ad78e7b4e: Deleting consensus metadata
I20260808 07:10:22.418563 852 catalog_manager.cc:4841] TS a42cd1f61a6c40bebe01f11ad78e7b4e (127.30.54.67:46137): tablet e95d564d8d1d4fa7881528f53270364e (table test-table [id=43e8bf6915ed4f05a5274ca2cf2bcc75]) successfully deleted
W20260808 07:10:22.424415 853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:40967 (error 108)
I20260808 07:10:22.424712 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 1175
I20260808 07:10:22.430974 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:22.424775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40967 (local address 127.0.0.1:43006)
0808 07:10:22.430857 (+ 6082us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:22.430919 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6051,"thread_start_us":80,"threads_started":1}
W20260808 07:10:22.433004 853 catalog_manager.cc:4584] TS 8fd4d830d298407093afc226147ee7d4 (127.30.54.65:40967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e95d564d8d1d4fa7881528f53270364e: Network error: Client connection negotiation failed: client connection to 127.30.54.65:40967: connect: Connection refused (error 111)
I20260808 07:10:22.437372 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 1379
I20260808 07:10:22.447420 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 828
2026-08-08T07:10:22Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5852 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:22 dist-test-slave-znh6 krb5kdc[1816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:22 dist-test-slave-znh6 krb5kdc[1816](info): set up 2 sockets
Aug 08 07:10:22 dist-test-slave-znh6 krb5kdc[1816](info): commencing operation
krb5kdc: starting...
W20260808 07:10:23.678350 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:23 dist-test-slave-znh6 krb5kdc[1816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:23.800076 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:38509
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:38509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:23.880604 1832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:23.880806 1832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:23.880829 1832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:23.881965 1832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:23.882017 1832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:23.882032 1832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:23.882045 1832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:23.882056 1832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:23.883572 1832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:38509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:38509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:23.883826 1832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:23.884071 1832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:23.889338 1840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:23.889389 1839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:23.889386 1843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:23.889423 1832 server_base.cc:1034] running on GCE node
I20260808 07:10:23.889847 1832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:23.890118 1832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:23.891296 1832 hybrid_clock.cc:648] HybridClock initialized: now 1786173023891286 us; error 33 us; skew 500 ppm
Aug 08 07:10:23 dist-test-slave-znh6 krb5kdc[1816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173023, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:23.893824 1832 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:23.894302 1832 webserver.cc:460] Webserver started at http://127.30.54.126:43293/ using document root <none> and password file <none>
I20260808 07:10:23.894522 1832 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:23.894588 1832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:23.894738 1832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:23.895820 1832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "d61172df438a4b2a8bf674f49009ef15"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-znh6"
server_key: "8f9a7f3b64aa4c1fa82e73550c78f948"
server_key_iv: "2277d7aa8f9ad21087ac3a5db027b73b"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.896184 1832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "d61172df438a4b2a8bf674f49009ef15"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-znh6"
server_key: "8f9a7f3b64aa4c1fa82e73550c78f948"
server_key_iv: "2277d7aa8f9ad21087ac3a5db027b73b"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.897505 1832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:23.898406 1856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.898578 1832 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:23.898675 1832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "d61172df438a4b2a8bf674f49009ef15"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-znh6"
server_key: "8f9a7f3b64aa4c1fa82e73550c78f948"
server_key_iv: "2277d7aa8f9ad21087ac3a5db027b73b"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.898752 1832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:23.909161 1832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:23.911378 1832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:23.911538 1832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:23.915618 1832 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:38509
I20260808 07:10:23.915644 1939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:38509 every 8 connection(s)
I20260808 07:10:23.915999 1832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:23.916890 1940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:23.919202 1940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15: Bootstrap starting.
I20260808 07:10:23.919811 1940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:23.920120 1940 log.cc:824] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:23.920812 1940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15: No bootstrap required, opened a new log
I20260808 07:10:23.922396 1940 raft_consensus.cc:348] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61172df438a4b2a8bf674f49009ef15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38509 } }
I20260808 07:10:23.922549 1940 raft_consensus.cc:374] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:23.922574 1940 raft_consensus.cc:729] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61172df438a4b2a8bf674f49009ef15, State: Initialized, Role: FOLLOWER
I20260808 07:10:23.922744 1940 consensus_queue.cc:260] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61172df438a4b2a8bf674f49009ef15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38509 } }
I20260808 07:10:23.922865 1940 raft_consensus.cc:388] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:23.922905 1940 raft_consensus.cc:482] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:23.922937 1940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:23.923504 1940 raft_consensus.cc:504] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61172df438a4b2a8bf674f49009ef15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38509 } }
I20260808 07:10:23.923619 1940 leader_election.cc:302] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d61172df438a4b2a8bf674f49009ef15; no voters:
I20260808 07:10:23.923805 1940 leader_election.cc:288] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:23.923848 1944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:23.924077 1940 sys_catalog.cc:564] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:23.924206 1944 raft_consensus.cc:686] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [term 1 LEADER]: Becoming Leader. State: Replica: d61172df438a4b2a8bf674f49009ef15, State: Running, Role: LEADER
I20260808 07:10:23.924307 1944 consensus_queue.cc:237] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61172df438a4b2a8bf674f49009ef15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38509 } }
I20260808 07:10:23.924741 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 1832
I20260808 07:10:23.924846 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:23.925063 30937 external_mini_cluster.cc:1428] Setting key a5b055114e8066358204597f2652d362
I20260808 07:10:23.925120 1948 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d61172df438a4b2a8bf674f49009ef15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61172df438a4b2a8bf674f49009ef15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38509 } } }
I20260808 07:10:23.925222 1948 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:23.925405 1949 sys_catalog.cc:455] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d61172df438a4b2a8bf674f49009ef15. Latest consensus state: current_term: 1 leader_uuid: "d61172df438a4b2a8bf674f49009ef15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d61172df438a4b2a8bf674f49009ef15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38509 } } }
I20260808 07:10:23.925537 1949 sys_catalog.cc:458] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:23.925729 1963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:23.926438 1963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:23.928692 1963 catalog_manager.cc:1347] Generated new cluster ID: cb7f5d633dd74ac1b6342fc23e2c8f28
I20260808 07:10:23.928747 1963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:23 dist-test-slave-znh6 krb5kdc[1816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:23.936837 1969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.925823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51164 (local address 127.30.54.126:38509)
0808 07:10:23.925986 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.925995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.926031 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:10:23.926153 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.926161 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.926229 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.926385 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:23.927211 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.928289 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.929368 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.929966 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.930066 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.932998 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:23.933002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:23.933003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:23.933017 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:23.935139 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:23.935547 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:23.935551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:23.935551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:23.935667 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:23.935884 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:23.935885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:23.935886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:23.936209 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:23.936378 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.936456 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.936613 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260808 07:10:23.938791 1963 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:23.939304 1963 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:23.948848 1963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d61172df438a4b2a8bf674f49009ef15: Generated new TSK 0
I20260808 07:10:23.949050 1963 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:10:23.982554 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38509
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:24.065286 1977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:24.065501 1977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:24.065524 1977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:24.066677 1977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:24.066721 1977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:24.066747 1977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:24.068313 1977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:24.068585 1977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:24.068841 1977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:24.073871 1988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:24.073845 1991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:24.073854 1984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:24.074020 1977 server_base.cc:1034] running on GCE node
I20260808 07:10:24.074301 1977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:24.074504 1977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:24.075644 1977 hybrid_clock.cc:648] HybridClock initialized: now 1786173024075620 us; error 44 us; skew 500 ppm
Aug 08 07:10:24 dist-test-slave-znh6 krb5kdc[1816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:24.078261 1977 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:10:24.078715 1977 webserver.cc:460] Webserver started at http://127.30.54.65:46781/ using document root <none> and password file <none>
I20260808 07:10:24.078969 1977 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:24.079016 1977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:24.079106 1977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:24.080061 1977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "d05c9afb7fe94c938b2349684e6fea25"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-znh6"
server_key: "b62da222344ff2ec77ba1d6392efde94"
server_key_iv: "523320c8c4751e7b7914d1537fcfe046"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.080355 1977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "d05c9afb7fe94c938b2349684e6fea25"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-znh6"
server_key: "b62da222344ff2ec77ba1d6392efde94"
server_key_iv: "523320c8c4751e7b7914d1537fcfe046"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.081612 1977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:24.082253 1999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.082430 1977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:24.082530 1977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "d05c9afb7fe94c938b2349684e6fea25"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-znh6"
server_key: "b62da222344ff2ec77ba1d6392efde94"
server_key_iv: "523320c8c4751e7b7914d1537fcfe046"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.082609 1977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:24.097729 1977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:24.100235 1977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:24.100409 1977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:24.100642 1977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:24.100953 1977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:24.101009 1977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.101052 1977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:24.101082 1977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.107173 1977 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:34335
I20260808 07:10:24.107210 2170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:34335 every 8 connection(s)
I20260808 07:10:24.107542 1977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:24.107954 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 1977
I20260808 07:10:24.108050 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:24.108244 30937 external_mini_cluster.cc:1428] Setting key 9c0788081e65d8c65d903749b8c5f4be
Aug 08 07:10:24 dist-test-slave-znh6 krb5kdc[1816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
I20260808 07:10:24.118469 1969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.108337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:54777 (local address 127.30.54.126:38509)
0808 07:10:24.108382 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.108385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.108627 (+ 242us) server_negotiation.cc:407] Connection header received
0808 07:10:24.109181 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.109182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.109194 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.109266 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:24.110189 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.111037 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.112542 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.112751 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.112803 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.115302 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:24.115304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:24.115304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:24.115312 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:24.117014 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:24.117412 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:24.117414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:24.117414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:24.117481 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:24.117735 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:24.117737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:24.117737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:24.117905 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:24.117984 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.118234 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.118342 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:10:24.118561 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.108296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:38509 (local address 127.30.54.65:54777)
0808 07:10:24.108489 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.108530 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.109077 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.109322 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.109326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.109900 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:24.110084 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.110089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.111136 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.111137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.112448 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.112452 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.112836 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.112837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.112996 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.113510 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:24.113521 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:24.115201 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:24.117103 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:24.117110 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:24.117111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:24.117329 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:24.117560 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:24.117561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:24.117561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:24.117660 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:24.117978 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:24.117982 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:24.118103 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.118183 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.118302 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":121,"threads_started":1}
I20260808 07:10:24.119149 2171 heartbeater.cc:347] Connected to a master server at 127.30.54.126:38509
I20260808 07:10:24.119239 2171 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:24.119378 2171 heartbeater.cc:511] Master 127.30.54.126:38509 requested a full tablet report, sending...
I20260808 07:10:24.119899 1877 ts_manager.cc:194] Registered new tserver with Master: d05c9afb7fe94c938b2349684e6fea25 (127.30.54.65:34335)
I20260808 07:10:24.120518 1877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:54777
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:10:24.138417 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38509
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:24.219384 2176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:24.219573 2176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:24.219597 2176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:24.220727 2176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:24.220769 2176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:24.220822 2176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:24.222222 2176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:24.222460 2176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:24.222681 2176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:24.227802 2184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:24.227830 2186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:24.227833 2183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:24.228142 2176 server_base.cc:1034] running on GCE node
I20260808 07:10:24.228344 2176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:24.228533 2176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:24.229686 2176 hybrid_clock.cc:648] HybridClock initialized: now 1786173024229678 us; error 36 us; skew 500 ppm
Aug 08 07:10:24 dist-test-slave-znh6 krb5kdc[1816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:24.232275 2176 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:10:24.232764 2176 webserver.cc:460] Webserver started at http://127.30.54.66:42419/ using document root <none> and password file <none>
I20260808 07:10:24.233017 2176 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:24.233088 2176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:24.233240 2176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:24.234243 2176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "6529919e81f54f10826bf71f1c09b06c"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-znh6"
server_key: "c2fb75ebdbe64d53470c1220ecc0fffe"
server_key_iv: "df650e5a47904de7d1ebeec93ecb4cb1"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.234619 2176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "6529919e81f54f10826bf71f1c09b06c"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-znh6"
server_key: "c2fb75ebdbe64d53470c1220ecc0fffe"
server_key_iv: "df650e5a47904de7d1ebeec93ecb4cb1"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.236020 2176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:24.236763 2195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.237039 2176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:24.237136 2176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "6529919e81f54f10826bf71f1c09b06c"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-znh6"
server_key: "c2fb75ebdbe64d53470c1220ecc0fffe"
server_key_iv: "df650e5a47904de7d1ebeec93ecb4cb1"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.237208 2176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:24.253995 2176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:24.255877 2176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:24.256034 2176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:24.256260 2176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:24.256561 2176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:24.256616 2176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.256656 2176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:24.256685 2176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.262661 2176 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:35855
I20260808 07:10:24.262707 2358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:35855 every 8 connection(s)
I20260808 07:10:24.263108 2176 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:24.263600 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 2176
I20260808 07:10:24.263705 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:24.263926 30937 external_mini_cluster.cc:1428] Setting key e8d15fc1f1cc67796d26380ac6ead5d4
Aug 08 07:10:24 dist-test-slave-znh6 krb5kdc[1816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
I20260808 07:10:24.273842 1969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.263857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:45023 (local address 127.30.54.126:38509)
0808 07:10:24.263938 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.263941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.264187 (+ 246us) server_negotiation.cc:407] Connection header received
0808 07:10:24.264876 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.264878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.264893 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.264969 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:24.265839 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.266626 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.268182 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.268378 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.268427 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.270985 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:24.270987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:24.270987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:24.270995 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:24.272434 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:24.272845 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:24.272847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:24.272847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:24.272910 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:24.273173 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:24.273174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:24.273175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:24.273294 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:24.273369 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.273656 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.273751 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:10:24.274019 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.263824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:38509 (local address 127.30.54.66:45023)
0808 07:10:24.264055 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.264103 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.264756 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.265019 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.265022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.265570 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:24.265748 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.265752 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.266728 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.266729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.268092 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.268095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.268457 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.268458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.268604 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.269096 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:24.269106 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:24.270892 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:24.272529 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:24.272537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:24.272539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:24.272764 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:24.272981 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:24.272982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:24.272982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:24.273095 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:24.273359 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:24.273366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:24.273514 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.273621 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.273783 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":123,"threads_started":1}
I20260808 07:10:24.274534 2359 heartbeater.cc:347] Connected to a master server at 127.30.54.126:38509
I20260808 07:10:24.274633 2359 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:24.274770 2359 heartbeater.cc:511] Master 127.30.54.126:38509 requested a full tablet report, sending...
I20260808 07:10:24.275182 1877 ts_manager.cc:194] Registered new tserver with Master: 6529919e81f54f10826bf71f1c09b06c (127.30.54.66:35855)
I20260808 07:10:24.275527 1877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:45023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:10:24.295339 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38509
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:24.390069 2365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:24.390268 2365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:24.390292 2365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:24.391451 2365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:24.391494 2365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:24.391520 2365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:24.392836 2365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:24.393002 2365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:24.393167 2365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:24.398260 2375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:24.398257 2377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:24.398444 2365 server_base.cc:1034] running on GCE node
W20260808 07:10:24.398466 2373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:24.398979 2365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:24.399272 2365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:24.406590 2365 hybrid_clock.cc:648] HybridClock initialized: now 1786173024406557 us; error 48 us; skew 500 ppm
Aug 08 07:10:24 dist-test-slave-znh6 krb5kdc[1816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:24.409616 2365 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:10:24.410120 2365 webserver.cc:460] Webserver started at http://127.30.54.67:40445/ using document root <none> and password file <none>
I20260808 07:10:24.410303 2365 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:24.410344 2365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:24.410436 2365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:24.411566 2365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "8447ccc5bf5c46c8860b9d4a5b34cbfe"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-znh6"
server_key: "e358a9167ecf5e623757ac71d4ab5091"
server_key_iv: "488459541f86002bbbb725137b06cce0"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.411866 2365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "8447ccc5bf5c46c8860b9d4a5b34cbfe"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-znh6"
server_key: "e358a9167ecf5e623757ac71d4ab5091"
server_key_iv: "488459541f86002bbbb725137b06cce0"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.413110 2365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:10:24.413748 2390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.413923 2365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:24.414016 2365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "8447ccc5bf5c46c8860b9d4a5b34cbfe"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-znh6"
server_key: "e358a9167ecf5e623757ac71d4ab5091"
server_key_iv: "488459541f86002bbbb725137b06cce0"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.414073 2365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:24.422139 2365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:24.424511 2365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:24.424616 2365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:24.424791 2365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:24.425192 2365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:24.425230 2365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.425251 2365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:24.425297 2365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.431128 2365 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:44207
I20260808 07:10:24.431208 2553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:44207 every 8 connection(s)
I20260808 07:10:24.431505 2365 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:10:24 dist-test-slave-znh6 krb5kdc[1816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:24.440233 1969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.432247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:35391 (local address 127.30.54.126:38509)
0808 07:10:24.432296 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.432298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.432488 (+ 190us) server_negotiation.cc:407] Connection header received
0808 07:10:24.432941 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.432942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.432951 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.433033 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:24.433902 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.434400 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.435377 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.435556 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.435611 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.437551 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:24.437553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:24.437553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:24.437560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:24.438986 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:24.439337 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:24.439338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:24.439339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:24.439390 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:24.439625 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:24.439625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:24.439626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:24.439725 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:24.439817 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.440073 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.440161 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:10:24.440372 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.432202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:38509 (local address 127.30.54.67:35391)
0808 07:10:24.432364 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.432405 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.432855 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.433104 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.433107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.433652 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:24.433828 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.433832 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.434476 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.434476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.435306 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.435308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.435629 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.435630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.435723 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.436129 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:24.436137 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:24.437461 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:24.439063 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:24.439069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:24.439071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:24.439274 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:24.439448 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:24.439449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:24.439449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:24.439571 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:24.439833 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:24.439839 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:24.439959 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.440042 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.440173 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260808 07:10:24.440922 2555 heartbeater.cc:347] Connected to a master server at 127.30.54.126:38509
I20260808 07:10:24.441015 2555 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:24.441208 2555 heartbeater.cc:511] Master 127.30.54.126:38509 requested a full tablet report, sending...
I20260808 07:10:24.441227 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 2365
I20260808 07:10:24.441300 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:24.441488 30937 external_mini_cluster.cc:1428] Setting key c972833c54e574481d7d865bfe817abb
I20260808 07:10:24.441630 1877 ts_manager.cc:194] Registered new tserver with Master: 8447ccc5bf5c46c8860b9d4a5b34cbfe (127.30.54.67:44207)
I20260808 07:10:24.442155 1877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:35391
I20260808 07:10:24.442289 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:10:24.456159 1969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:10:24.455250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51172 (local address 127.30.54.126:38509)
0808 07:10:24.455315 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.455317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.455434 (+ 117us) server_negotiation.cc:407] Connection header received
0808 07:10:24.455482 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.455483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.455492 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.455550 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:24.456087 (+ 537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51172: BlockingRecv error: recv got EOF from 127.0.0.1:51172 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:10:24.457031 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 1977
I20260808 07:10:24.466279 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 2176
I20260808 07:10:24.473793 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 2365
I20260808 07:10:24.481604 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 1832
2026-08-08T07:10:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2032 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:24 dist-test-slave-znh6 krb5kdc[2571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:24 dist-test-slave-znh6 krb5kdc[2571](info): set up 2 sockets
Aug 08 07:10:24 dist-test-slave-znh6 krb5kdc[2571](info): commencing operation
krb5kdc: starting...
W20260808 07:10:25.698505 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:25 dist-test-slave-znh6 krb5kdc[2571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:25.818599 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43115
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:43115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:25.900805 2590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:25.901008 2590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:25.901036 2590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:25.902175 2590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:25.902217 2590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:25.902231 2590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:25.902244 2590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:25.902302 2590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:25.903981 2590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:43115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:25.904241 2590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:25.904495 2590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:25.909211 2603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:25.909248 2600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:25.909209 2601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:25.909472 2590 server_base.cc:1034] running on GCE node
I20260808 07:10:25.909775 2590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:25.910063 2590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:25.911267 2590 hybrid_clock.cc:648] HybridClock initialized: now 1786173025911243 us; error 37 us; skew 500 ppm
Aug 08 07:10:25 dist-test-slave-znh6 krb5kdc[2571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173025, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:25.917603 2590 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:25.918020 2590 webserver.cc:460] Webserver started at http://127.30.54.126:36107/ using document root <none> and password file <none>
I20260808 07:10:25.918203 2590 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:25.918246 2590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:25.918342 2590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:25.919407 2590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "4f32494ecc10481d809191ca24233865"
format_stamp: "Formatted at 2026-08-08 07:10:25 on dist-test-slave-znh6"
server_key: "3b5d94115bda0bca068ca15361016a4d"
server_key_iv: "47cf370dc7846bef456f766f07552cf4"
server_key_version: "encryptionkey@0"
I20260808 07:10:25.919705 2590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "4f32494ecc10481d809191ca24233865"
format_stamp: "Formatted at 2026-08-08 07:10:25 on dist-test-slave-znh6"
server_key: "3b5d94115bda0bca068ca15361016a4d"
server_key_iv: "47cf370dc7846bef456f766f07552cf4"
server_key_version: "encryptionkey@0"
I20260808 07:10:25.921000 2590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:25.921733 2612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:25.921943 2590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:25.922029 2590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "4f32494ecc10481d809191ca24233865"
format_stamp: "Formatted at 2026-08-08 07:10:25 on dist-test-slave-znh6"
server_key: "3b5d94115bda0bca068ca15361016a4d"
server_key_iv: "47cf370dc7846bef456f766f07552cf4"
server_key_version: "encryptionkey@0"
I20260808 07:10:25.922083 2590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:25.930675 2590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:25.932909 2590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:25.933022 2590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:25.936915 2590 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:43115
I20260808 07:10:25.936966 2691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:43115 every 8 connection(s)
I20260808 07:10:25.937482 2590 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:25.938186 2693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:25.940570 2693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865: Bootstrap starting.
I20260808 07:10:25.941174 2693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:25.941490 2693 log.cc:824] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:25.942176 2693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865: No bootstrap required, opened a new log
I20260808 07:10:25.943701 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 2590
I20260808 07:10:25.943822 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:25.944008 30937 external_mini_cluster.cc:1428] Setting key 1177be3b71f021e02ca68b794b2b4067
I20260808 07:10:25.944147 2693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f32494ecc10481d809191ca24233865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43115 } }
I20260808 07:10:25.944322 2693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:25.944370 2693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f32494ecc10481d809191ca24233865, State: Initialized, Role: FOLLOWER
I20260808 07:10:25.944480 2693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f32494ecc10481d809191ca24233865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43115 } }
I20260808 07:10:25.944561 2693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:25.944599 2693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:25.944641 2693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:25.945399 2693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f32494ecc10481d809191ca24233865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43115 } }
I20260808 07:10:25.945654 2693 leader_election.cc:302] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f32494ecc10481d809191ca24233865; no voters:
I20260808 07:10:25.945868 2693 leader_election.cc:288] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:25.945904 2699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:25.946022 2699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [term 1 LEADER]: Becoming Leader. State: Replica: 4f32494ecc10481d809191ca24233865, State: Running, Role: LEADER
I20260808 07:10:25.946107 2699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f32494ecc10481d809191ca24233865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43115 } }
I20260808 07:10:25.946154 2693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:25.946429 2699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f32494ecc10481d809191ca24233865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f32494ecc10481d809191ca24233865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43115 } } }
I20260808 07:10:25.946504 2699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:25.946491 2702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f32494ecc10481d809191ca24233865. Latest consensus state: current_term: 1 leader_uuid: "4f32494ecc10481d809191ca24233865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f32494ecc10481d809191ca24233865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43115 } } }
I20260808 07:10:25.946596 2702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:25.946803 2706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:25.947346 2706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:25.949218 2706 catalog_manager.cc:1347] Generated new cluster ID: 6911c7198048432eb3924c1d02bf2752
I20260808 07:10:25.949275 2706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:25 dist-test-slave-znh6 krb5kdc[2571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:25.956421 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:25.944484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49840 (local address 127.30.54.126:43115)
0808 07:10:25.945160 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:25.945169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:25.945205 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:10:25.945322 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:25.945335 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:25.945376 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:25.945553 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:25.946441 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:25.948128 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:25.949162 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:25.949757 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:25.949865 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:25.952685 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:25.952690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:25.952691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:25.952703 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:25.954743 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:25.955151 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:25.955155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:25.955155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:25.955282 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:25.955494 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:25.955496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:25.955496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:25.955800 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:25.955976 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:25.956045 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:10:25.956189 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":550,"threads_started":1}
I20260808 07:10:25.957450 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:25.960541 2706 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:25.961293 2706 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:25.965219 2706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f32494ecc10481d809191ca24233865: Generated new TSK 0
I20260808 07:10:25.965408 2706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:25.973865 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:25.970168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49856 (local address 127.30.54.126:43115)
0808 07:10:25.970209 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:25.970211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:25.970347 (+ 136us) server_negotiation.cc:407] Connection header received
0808 07:10:25.970378 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:25.970379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:25.970421 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:25.970506 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:25.970870 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:25.971643 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:25.972364 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:25.972697 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:25.972767 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:25.973392 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:25.973393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:25.973393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:25.973620 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:25.973637 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:25.973644 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:25.973706 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:25.973757 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 07:10:25.973784 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:10:25.976207 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 2590
2026-08-08T07:10:25Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1492 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:26 dist-test-slave-znh6 krb5kdc[2740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:26 dist-test-slave-znh6 krb5kdc[2740](info): set up 2 sockets
Aug 08 07:10:26 dist-test-slave-znh6 krb5kdc[2740](info): commencing operation
krb5kdc: starting...
W20260808 07:10:27.202589 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:27.322676 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:35273
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:35273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:27.404119 2765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:27.404350 2765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:27.404390 2765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:27.405560 2765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:27.405630 2765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:27.405653 2765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:27.405674 2765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:27.405705 2765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:27.407238 2765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:35273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:35273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:27.407512 2765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:27.407739 2765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:27.412608 2772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.412617 2774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.412652 2765 server_base.cc:1034] running on GCE node
W20260808 07:10:27.412628 2779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.413093 2765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:27.413378 2765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:27.414520 2765 hybrid_clock.cc:648] HybridClock initialized: now 1786173027414509 us; error 30 us; skew 500 ppm
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:27.417061 2765 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:27.417448 2765 webserver.cc:460] Webserver started at http://127.30.54.126:35013/ using document root <none> and password file <none>
I20260808 07:10:27.417621 2765 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:27.417663 2765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:27.417749 2765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:27.418720 2765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "9579d4a376654cb99b38bcd099389c15"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "be0db6128eacc14708c0f1c7c634335c"
server_key_iv: "079132dc3454039a5ae44eb587a59742"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.419101 2765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "9579d4a376654cb99b38bcd099389c15"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "be0db6128eacc14708c0f1c7c634335c"
server_key_iv: "079132dc3454039a5ae44eb587a59742"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.420393 2765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:27.421121 2790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.421291 2765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:27.421391 2765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "9579d4a376654cb99b38bcd099389c15"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "be0db6128eacc14708c0f1c7c634335c"
server_key_iv: "079132dc3454039a5ae44eb587a59742"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.421450 2765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:27.431205 2765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:27.433017 2765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:27.433125 2765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:27.436993 2765 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:35273
I20260808 07:10:27.437047 2865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:35273 every 8 connection(s)
I20260808 07:10:27.437400 2765 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:27.438426 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:27.440769 2866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15: Bootstrap starting.
I20260808 07:10:27.441349 2866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:27.441648 2866 log.cc:824] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:27.442410 2866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15: No bootstrap required, opened a new log
I20260808 07:10:27.444140 2866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9579d4a376654cb99b38bcd099389c15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35273 } }
I20260808 07:10:27.444301 2866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:27.444347 2866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9579d4a376654cb99b38bcd099389c15, State: Initialized, Role: FOLLOWER
I20260808 07:10:27.444540 2866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9579d4a376654cb99b38bcd099389c15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35273 } }
I20260808 07:10:27.444646 2866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:27.444692 2866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:27.444747 2866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:27.445564 2866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9579d4a376654cb99b38bcd099389c15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35273 } }
I20260808 07:10:27.445688 2866 leader_election.cc:302] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9579d4a376654cb99b38bcd099389c15; no voters:
I20260808 07:10:27.445889 2866 leader_election.cc:288] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:27.445950 2869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:27.446152 2866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:27.446162 2869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [term 1 LEADER]: Becoming Leader. State: Replica: 9579d4a376654cb99b38bcd099389c15, State: Running, Role: LEADER
I20260808 07:10:27.446281 2869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9579d4a376654cb99b38bcd099389c15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35273 } }
I20260808 07:10:27.446736 2871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9579d4a376654cb99b38bcd099389c15. Latest consensus state: current_term: 1 leader_uuid: "9579d4a376654cb99b38bcd099389c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9579d4a376654cb99b38bcd099389c15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35273 } } }
I20260808 07:10:27.446843 2871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:27.447073 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 2765
I20260808 07:10:27.447098 2881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:27.447180 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:27.447074 2870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9579d4a376654cb99b38bcd099389c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9579d4a376654cb99b38bcd099389c15" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35273 } } }
I20260808 07:10:27.447368 30937 external_mini_cluster.cc:1428] Setting key 94279c38a486eb6d22eadbedec1e1976
I20260808 07:10:27.447371 2870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:27.447626 2881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:27.449326 2881 catalog_manager.cc:1347] Generated new cluster ID: 47e584c351bd49d1ad722efbfa3a3858
I20260808 07:10:27.449388 2881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:27.459878 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.447839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.30.54.126:35273)
0808 07:10:27.448124 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.448134 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.448168 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:10:27.448349 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.448358 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.448417 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.448549 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.449343 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.450707 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.451860 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.452522 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.452601 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.455587 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.455592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.455593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:27.455605 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.458176 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.458580 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.458583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.458584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.458719 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.458955 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.458956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.458956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.459267 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.459442 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.459520 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.459655 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":75,"threads_started":1}
I20260808 07:10:27.464196 2881 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:27.464666 2881 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:27.469355 2881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9579d4a376654cb99b38bcd099389c15: Generated new TSK 0
I20260808 07:10:27.469532 2881 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:10:27.498163 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35273
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:27.579722 2909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:27.579911 2909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:27.579933 2909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:27.580991 2909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:27.581027 2909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:27.581051 2909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:27.582562 2909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:27.582798 2909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:27.582993 2909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:27.587957 2919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.587965 2921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.587996 2909 server_base.cc:1034] running on GCE node
W20260808 07:10:27.587977 2917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.588406 2909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:27.588682 2909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:27.589843 2909 hybrid_clock.cc:648] HybridClock initialized: now 1786173027589826 us; error 48 us; skew 500 ppm
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:27.592450 2909 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:10:27.592948 2909 webserver.cc:460] Webserver started at http://127.30.54.65:42805/ using document root <none> and password file <none>
I20260808 07:10:27.593122 2909 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:27.593160 2909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:27.593245 2909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:27.594182 2909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "58c75355335445d5b55dd8d33b7791e8"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "bcb3b8198aa20ee50436e822d866b85b"
server_key_iv: "ec6ff2653a3eb4b94918966a5358248c"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.594492 2909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "58c75355335445d5b55dd8d33b7791e8"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "bcb3b8198aa20ee50436e822d866b85b"
server_key_iv: "ec6ff2653a3eb4b94918966a5358248c"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.595805 2909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:27.596465 2929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.596655 2909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:27.596735 2909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "58c75355335445d5b55dd8d33b7791e8"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "bcb3b8198aa20ee50436e822d866b85b"
server_key_iv: "ec6ff2653a3eb4b94918966a5358248c"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.596786 2909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:27.608330 2909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:27.610505 2909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:27.610612 2909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:27.610790 2909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:27.611104 2909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:27.611138 2909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.611159 2909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:27.611172 2909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.616927 2909 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:38561
I20260808 07:10:27.617005 3089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:38561 every 8 connection(s)
I20260808 07:10:27.617304 2909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:27.623274 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 2909
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:27.623381 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:27.623615 30937 external_mini_cluster.cc:1428] Setting key 96999233a08824cf2e1cc208f24c9271
I20260808 07:10:27.626969 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.618027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:49051 (local address 127.30.54.126:35273)
0808 07:10:27.618074 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.618077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.618247 (+ 170us) server_negotiation.cc:407] Connection header received
0808 07:10:27.618770 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.618772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.618790 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.618959 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.619898 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.620415 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.621415 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.621634 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.621724 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.623806 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.623807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.623808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:27.623816 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.625409 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.625898 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.625899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.625900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.625952 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.626259 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.626260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.626261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.626418 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.626494 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.626726 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.626848 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:10:27.627007 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.617993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:35273 (local address 127.30.54.65:49051)
0808 07:10:27.618126 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.618167 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.618662 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.619039 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.619043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.619595 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:27.619769 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.619773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.620520 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.620521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.621320 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.621322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.621735 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.621736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.621862 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.622267 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:27.622275 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:27.623701 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:27.625481 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:27.625489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:27.625490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:27.625815 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:27.626022 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:27.626023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:27.626023 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:27.626177 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:27.626489 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:27.626494 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:27.626599 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.626682 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.626793 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260808 07:10:27.627480 3090 heartbeater.cc:347] Connected to a master server at 127.30.54.126:35273
I20260808 07:10:27.627576 3090 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:27.627759 3090 heartbeater.cc:511] Master 127.30.54.126:35273 requested a full tablet report, sending...
I20260808 07:10:27.628244 2813 ts_manager.cc:194] Registered new tserver with Master: 58c75355335445d5b55dd8d33b7791e8 (127.30.54.65:38561)
I20260808 07:10:27.628891 2813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:49051
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:10:27.653384 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35273
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:27.736472 3096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:27.736663 3096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:27.736686 3096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:27.737825 3096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:27.737865 3096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:27.737891 3096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:27.739318 3096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:27.739550 3096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:27.739771 3096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:27.745075 3103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.745205 3096 server_base.cc:1034] running on GCE node
W20260808 07:10:27.745054 3106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.745031 3102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.745560 3096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:27.745797 3096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:27.746899 3096 hybrid_clock.cc:648] HybridClock initialized: now 1786173027746864 us; error 46 us; skew 500 ppm
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:27.749516 3096 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:10:27.749882 3096 webserver.cc:460] Webserver started at http://127.30.54.66:37457/ using document root <none> and password file <none>
I20260808 07:10:27.750106 3096 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:27.750176 3096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:27.750326 3096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:27.751376 3096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "0c36dc8068e542d7a5bcf4dbed6df336"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "3cfec09654c932a95102de1de3b48ad1"
server_key_iv: "2d1a197daae8fc5aa64c6ac1eb23238a"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.751735 3096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "0c36dc8068e542d7a5bcf4dbed6df336"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "3cfec09654c932a95102de1de3b48ad1"
server_key_iv: "2d1a197daae8fc5aa64c6ac1eb23238a"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.753067 3096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:27.753808 3115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.753983 3096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:27.754082 3096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "0c36dc8068e542d7a5bcf4dbed6df336"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "3cfec09654c932a95102de1de3b48ad1"
server_key_iv: "2d1a197daae8fc5aa64c6ac1eb23238a"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.754160 3096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:27.766932 3096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:27.768702 3096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:27.768858 3096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:27.769078 3096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:27.769381 3096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:27.769439 3096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.769482 3096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:27.769510 3096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.775484 3096 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:34941
I20260808 07:10:27.775537 3279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:34941 every 8 connection(s)
I20260808 07:10:27.775864 3096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:27.778626 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 3096
I20260808 07:10:27.778733 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:27.778960 30937 external_mini_cluster.cc:1428] Setting key 16d4eabc7ee318837b28f437c99ea0fb
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:27.785739 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.776587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:45415 (local address 127.30.54.126:35273)
0808 07:10:27.776659 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.776662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.776837 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:10:27.777327 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.777329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.777340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.777412 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.778329 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.779184 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.780206 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.780411 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.780492 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.782858 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.782860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.782860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:27.782867 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.784345 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.784743 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.784744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.784745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.784808 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.785077 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.785078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.785078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.785191 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.785265 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.785553 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.785648 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:10:27.785895 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.776562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:35273 (local address 127.30.54.66:45415)
0808 07:10:27.776725 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.776758 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.777215 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.777490 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.777493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.778056 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:27.778227 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.778232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.779289 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.779291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.780093 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.780095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.780499 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.780501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.780630 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.781134 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:27.781142 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:27.782730 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:27.784432 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:27.784439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:27.784441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:27.784661 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:27.784891 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:27.784892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:27.784892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:27.785000 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:27.785267 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:27.785275 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:27.785414 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.785518 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.785672 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260808 07:10:27.786376 3282 heartbeater.cc:347] Connected to a master server at 127.30.54.126:35273
I20260808 07:10:27.786466 3282 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:27.786657 3282 heartbeater.cc:511] Master 127.30.54.126:35273 requested a full tablet report, sending...
I20260808 07:10:27.787075 2813 ts_manager.cc:194] Registered new tserver with Master: 0c36dc8068e542d7a5bcf4dbed6df336 (127.30.54.66:34941)
I20260808 07:10:27.787693 2813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:45415
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:10:27.805680 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35273
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:27.885985 3289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:27.886178 3289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:27.886200 3289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:27.887379 3289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:27.887422 3289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:27.887449 3289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:27.888883 3289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:27.889117 3289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:27.889344 3289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:27.894366 3298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.894392 3295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.894445 3296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.894863 3289 server_base.cc:1034] running on GCE node
I20260808 07:10:27.895071 3289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:27.895313 3289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:27.896453 3289 hybrid_clock.cc:648] HybridClock initialized: now 1786173027896439 us; error 30 us; skew 500 ppm
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:27.898943 3289 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:10:27.899324 3289 webserver.cc:460] Webserver started at http://127.30.54.67:41535/ using document root <none> and password file <none>
I20260808 07:10:27.899525 3289 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:27.899565 3289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:27.899652 3289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:27.900676 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "e7f394e8abaf451e95d0cd423b4e753e"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "f34c604ef851f6f4a1770658d64692a7"
server_key_iv: "8e260ac89d1f313ce40f59b298e30971"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.900971 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "e7f394e8abaf451e95d0cd423b4e753e"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "f34c604ef851f6f4a1770658d64692a7"
server_key_iv: "8e260ac89d1f313ce40f59b298e30971"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.902197 3289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:27.902923 3307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.903079 3289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:27.903177 3289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "e7f394e8abaf451e95d0cd423b4e753e"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-znh6"
server_key: "f34c604ef851f6f4a1770658d64692a7"
server_key_iv: "8e260ac89d1f313ce40f59b298e30971"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.903230 3289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:27.919888 3289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:27.921757 3289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:27.921864 3289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:27.922046 3289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:27.922297 3289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:27.922328 3289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.922349 3289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:27.922361 3289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.928375 3289 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:33849
I20260808 07:10:27.928447 3476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:33849 every 8 connection(s)
I20260808 07:10:27.928743 3289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:27.930358 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 3289
I20260808 07:10:27.930497 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:27.930733 30937 external_mini_cluster.cc:1428] Setting key d9664a64d27bdcde8b5d2c72fc6cb88d
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:27.938537 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.929441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:41317 (local address 127.30.54.126:35273)
0808 07:10:27.929484 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.929486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.929688 (+ 202us) server_negotiation.cc:407] Connection header received
0808 07:10:27.930125 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.930126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.930136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.930191 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.931368 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.931991 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.933308 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.933686 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.933750 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.935901 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.935903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.935903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:27.935911 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.937156 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.937563 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.937565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.937565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.937625 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.937891 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.937892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.937893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.938014 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.938086 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.938356 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.938446 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:10:27.938712 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.929406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:35273 (local address 127.30.54.67:41317)
0808 07:10:27.929576 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.929608 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.930045 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.930299 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.930303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.931058 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:27.931270 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.931277 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.932069 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.932069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.933199 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.933202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.933771 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.933772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.933907 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.934350 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:27.934357 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:27.935793 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:27.937255 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:27.937261 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:27.937263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:27.937485 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:27.937701 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:27.937702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:27.937702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:27.937811 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:27.938088 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:27.938095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:27.938225 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.938332 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.938483 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":114,"threads_started":1}
I20260808 07:10:27.939285 3477 heartbeater.cc:347] Connected to a master server at 127.30.54.126:35273
I20260808 07:10:27.939388 3477 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:27.939536 3477 heartbeater.cc:511] Master 127.30.54.126:35273 requested a full tablet report, sending...
I20260808 07:10:27.939922 2813 ts_manager.cc:194] Registered new tserver with Master: e7f394e8abaf451e95d0cd423b4e753e (127.30.54.67:33849)
I20260808 07:10:27.940299 2813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:41317
I20260808 07:10:27.942468 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 07:10:27 dist-test-slave-znh6 krb5kdc[2740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173027, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:27.966804 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.959921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40594 (local address 127.30.54.126:35273)
0808 07:10:27.959986 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.959988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.960075 (+ 87us) server_negotiation.cc:407] Connection header received
0808 07:10:27.960115 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.960116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.960125 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.960179 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.960727 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.961193 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.961898 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.962078 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.962135 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.964483 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.964484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.964485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:27.964491 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.965704 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.966027 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.966028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.966028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.966077 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:27.966241 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:27.966242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:27.966242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:27.966396 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.966472 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.966621 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.966679 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 07:10:27.967087 2813 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40594
I20260808 07:10:27.967608 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 2909
I20260808 07:10:27.978477 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 3096
I20260808 07:10:27.989825 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 3289
I20260808 07:10:27.999186 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 2765
2026-08-08T07:10:28Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2024 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:28 dist-test-slave-znh6 krb5kdc[3495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:28 dist-test-slave-znh6 krb5kdc[3495](info): set up 2 sockets
Aug 08 07:10:28 dist-test-slave-znh6 krb5kdc[3495](info): commencing operation
krb5kdc: starting...
W20260808 07:10:29.216565 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:29.336055 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41991
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:41991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:29.417636 3512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:29.417874 3512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:29.417912 3512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:29.419122 3512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:29.419184 3512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:29.419207 3512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:29.419236 3512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:29.419268 3512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:29.420749 3512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:41991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:29.421020 3512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:29.421240 3512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:29.426127 3521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.426158 3520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.426177 3523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.426339 3512 server_base.cc:1034] running on GCE node
I20260808 07:10:29.426556 3512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:29.426856 3512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:29.428009 3512 hybrid_clock.cc:648] HybridClock initialized: now 1786173029427999 us; error 28 us; skew 500 ppm
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:29.430462 3512 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:29.430936 3512 webserver.cc:460] Webserver started at http://127.30.54.126:35737/ using document root <none> and password file <none>
I20260808 07:10:29.431154 3512 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:29.431222 3512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:29.431361 3512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:29.432356 3512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "ac66661800124efbbaf3acbf7862779f"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "276a9da5d4ac7db8474840576d2413a4"
server_key_iv: "a3d8b646d266680f1307684dc67afcdd"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.432715 3512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "ac66661800124efbbaf3acbf7862779f"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "276a9da5d4ac7db8474840576d2413a4"
server_key_iv: "a3d8b646d266680f1307684dc67afcdd"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.434018 3512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:29.434746 3534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.434948 3512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:29.435048 3512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "ac66661800124efbbaf3acbf7862779f"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "276a9da5d4ac7db8474840576d2413a4"
server_key_iv: "a3d8b646d266680f1307684dc67afcdd"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.435129 3512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:29.443173 3512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:29.445389 3512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:29.445559 3512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:29.449700 3512 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:41991
I20260808 07:10:29.449730 3602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:41991 every 8 connection(s)
I20260808 07:10:29.450212 3512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:29.450699 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 3512
I20260808 07:10:29.450805 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:29.450966 3603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:29.451004 30937 external_mini_cluster.cc:1428] Setting key 0d40b78ffe8657926d626a7d470e398e
I20260808 07:10:29.454198 3603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f: Bootstrap starting.
I20260808 07:10:29.454898 3603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:29.455236 3603 log.cc:824] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:29.456013 3603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f: No bootstrap required, opened a new log
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:29.458177 3603 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac66661800124efbbaf3acbf7862779f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41991 } }
I20260808 07:10:29.458343 3603 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:29.458408 3603 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac66661800124efbbaf3acbf7862779f, State: Initialized, Role: FOLLOWER
I20260808 07:10:29.458535 3603 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac66661800124efbbaf3acbf7862779f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41991 } }
I20260808 07:10:29.458606 3603 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:29.458629 3603 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:29.458702 3603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:29.459411 3603 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac66661800124efbbaf3acbf7862779f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41991 } }
I20260808 07:10:29.459544 3603 leader_election.cc:302] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac66661800124efbbaf3acbf7862779f; no voters:
I20260808 07:10:29.459702 3603 leader_election.cc:288] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:29.459739 3610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:29.459843 3610 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [term 1 LEADER]: Becoming Leader. State: Replica: ac66661800124efbbaf3acbf7862779f, State: Running, Role: LEADER
I20260808 07:10:29.459925 3610 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac66661800124efbbaf3acbf7862779f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41991 } }
I20260808 07:10:29.459964 3603 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:29.460242 3610 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac66661800124efbbaf3acbf7862779f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac66661800124efbbaf3acbf7862779f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41991 } } }
I20260808 07:10:29.460407 3610 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:29.460609 3610 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac66661800124efbbaf3acbf7862779f. Latest consensus state: current_term: 1 leader_uuid: "ac66661800124efbbaf3acbf7862779f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac66661800124efbbaf3acbf7862779f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41991 } } }
I20260808 07:10:29.460696 3610 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:29.461086 3620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:29.461656 3620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:29.462267 3606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.451511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.30.54.126:41991)
0808 07:10:29.451692 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.451702 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.451734 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:10:29.452148 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.452156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.452217 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.452372 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.452934 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.454312 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.455165 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.455709 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.455820 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.458334 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.458355 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.458355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.458367 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.460466 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.460943 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.460946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.460946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.461056 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.461301 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.461302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.461302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.461590 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.461709 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.461837 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.461986 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260808 07:10:29.463565 3620 catalog_manager.cc:1347] Generated new cluster ID: 69e744f6c7704f7384cdfb318a664c0f
I20260808 07:10:29.463632 3620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:29.478946 3620 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:29.479683 3620 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:29.484175 3620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac66661800124efbbaf3acbf7862779f: Generated new TSK 0
I20260808 07:10:29.484331 3620 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:10:29.519125 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41991
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:29.601123 3637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:29.601354 3637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:29.601394 3637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:29.602535 3637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:29.602599 3637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:29.602645 3637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:29.604215 3637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:29.604496 3637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:29.604729 3637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:29.609653 3643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.609697 3642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.609651 3646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.609959 3637 server_base.cc:1034] running on GCE node
I20260808 07:10:29.610234 3637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:29.610473 3637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:29.611637 3637 hybrid_clock.cc:648] HybridClock initialized: now 1786173029611602 us; error 45 us; skew 500 ppm
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:29.614068 3637 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:10:29.614507 3637 webserver.cc:460] Webserver started at http://127.30.54.65:35227/ using document root <none> and password file <none>
I20260808 07:10:29.614749 3637 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:29.614823 3637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:29.614993 3637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:29.615971 3637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "a779f385534e4c60b1c332bfed1ded4c"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "a1f668582f6f128df223870636e58b40"
server_key_iv: "364974cf82a7a1a3e0b206d5d8859603"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.616335 3637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "a779f385534e4c60b1c332bfed1ded4c"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "a1f668582f6f128df223870636e58b40"
server_key_iv: "364974cf82a7a1a3e0b206d5d8859603"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.617664 3637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:29.618377 3655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.618561 3637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:29.618649 3637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "a779f385534e4c60b1c332bfed1ded4c"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "a1f668582f6f128df223870636e58b40"
server_key_iv: "364974cf82a7a1a3e0b206d5d8859603"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.618729 3637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:29.628001 3637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:29.629658 3637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:29.629815 3637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:29.630034 3637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:29.630335 3637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:29.630391 3637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.630431 3637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:29.630458 3637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.636198 3637 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:45283
I20260808 07:10:29.636233 3818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:45283 every 8 connection(s)
I20260808 07:10:29.636554 3637 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:29.644596 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 3637
I20260808 07:10:29.644706 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:29.644907 30937 external_mini_cluster.cc:1428] Setting key 8bdc4272054538a7d809ad2c1ccfa16a
I20260808 07:10:29.646520 3606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.637388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:54307 (local address 127.30.54.126:41991)
0808 07:10:29.637471 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.637473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.637649 (+ 176us) server_negotiation.cc:407] Connection header received
0808 07:10:29.638716 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.638717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.638730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.638783 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.639501 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.640027 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.641045 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.641262 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.641341 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.643605 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.643607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.643607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.643615 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.645140 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.645542 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.645543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.645544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.645606 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.645867 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.645868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.645869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.645987 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.646059 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.646338 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.646428 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:10:29.646710 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.637331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41991 (local address 127.30.54.65:54307)
0808 07:10:29.637520 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:29.637566 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:29.638608 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:29.638898 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:29.638901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:29.639209 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:29.639393 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.639396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.640127 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.640128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.640943 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.640945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.641348 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.641350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.641484 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.642001 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:29.642009 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:29.643502 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:29.645225 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.645232 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.645234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.645461 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.645685 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.645686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.645686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.645791 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.646063 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:29.646071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:29.646202 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 07:10:29.646305 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 07:10:29.646470 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260808 07:10:29.647254 3819 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41991
I20260808 07:10:29.647334 3819 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:29.647519 3819 heartbeater.cc:511] Master 127.30.54.126:41991 requested a full tablet report, sending...
I20260808 07:10:29.648140 3557 ts_manager.cc:194] Registered new tserver with Master: a779f385534e4c60b1c332bfed1ded4c (127.30.54.65:45283)
I20260808 07:10:29.649154 3557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:54307
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:10:29.675132 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41991
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:29.755100 3824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:29.755288 3824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:29.755311 3824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:29.756368 3824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:29.756402 3824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:29.756426 3824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:29.758038 3824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:29.758315 3824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:29.758579 3824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:29.763685 3830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.763690 3831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.763964 3824 server_base.cc:1034] running on GCE node
W20260808 07:10:29.763700 3835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.764216 3824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:29.764487 3824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:29.765661 3824 hybrid_clock.cc:648] HybridClock initialized: now 1786173029765646 us; error 34 us; skew 500 ppm
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:29.768150 3824 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:10:29.768618 3824 webserver.cc:460] Webserver started at http://127.30.54.66:34857/ using document root <none> and password file <none>
I20260808 07:10:29.768846 3824 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:29.768890 3824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:29.769016 3824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:29.770056 3824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "6cc4ed4b0d394f42be81ccb9808d7610"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "61c69f17f21710ca8a2997355d2aba1b"
server_key_iv: "276382ae2a9748a6289e054f54073e85"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.770424 3824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "6cc4ed4b0d394f42be81ccb9808d7610"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "61c69f17f21710ca8a2997355d2aba1b"
server_key_iv: "276382ae2a9748a6289e054f54073e85"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.771829 3824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:29.772617 3844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.772796 3824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:10:29.772898 3824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "6cc4ed4b0d394f42be81ccb9808d7610"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "61c69f17f21710ca8a2997355d2aba1b"
server_key_iv: "276382ae2a9748a6289e054f54073e85"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.772976 3824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:29.784899 3824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:29.786792 3824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:29.786978 3824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:29.787199 3824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:29.787509 3824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:29.787573 3824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.787616 3824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:29.787642 3824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.794152 3824 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:42155
I20260808 07:10:29.794191 4005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:42155 every 8 connection(s)
I20260808 07:10:29.794518 3824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:29.800037 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 3824
I20260808 07:10:29.800150 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:29.800321 30937 external_mini_cluster.cc:1428] Setting key 4becb53dd83d3ae0a003bd1f77009031
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:29.803800 3606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.795250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:50515 (local address 127.30.54.126:41991)
0808 07:10:29.795297 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.795299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.795474 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:10:29.796386 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.796387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.796396 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.796463 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.797000 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.797501 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.798425 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.798612 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.798663 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.800982 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.800983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.800983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.800989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.802362 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.802833 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.802834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.802835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.802892 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.803156 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.803157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.803157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.803282 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.803353 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.803629 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.803712 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:29.803973 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.795200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41991 (local address 127.30.54.66:50515)
0808 07:10:29.795361 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:29.795400 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:29.796285 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:29.796516 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:29.796519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:29.796746 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:29.796921 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.796924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.797579 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.797580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.798352 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.798354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.798683 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.798684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.798778 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.799256 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:29.799264 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:29.800853 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:29.802494 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.802501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.802503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.802722 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.802972 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.802973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.802973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.803083 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.803359 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:29.803365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:29.803494 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 07:10:29.803594 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 07:10:29.803742 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260808 07:10:29.804535 4007 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41991
I20260808 07:10:29.804633 4007 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:29.804811 4007 heartbeater.cc:511] Master 127.30.54.126:41991 requested a full tablet report, sending...
I20260808 07:10:29.805199 3557 ts_manager.cc:194] Registered new tserver with Master: 6cc4ed4b0d394f42be81ccb9808d7610 (127.30.54.66:42155)
I20260808 07:10:29.805579 3557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:50515
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:10:29.827394 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41991
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:42797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:29.907645 4018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:29.907836 4018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:29.907859 4018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:29.908982 4018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:29.909024 4018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:29.909049 4018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:29.910540 4018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:42797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:29.910763 4018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:29.911016 4018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:29.915907 4030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.915905 4024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.915905 4025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.916146 4018 server_base.cc:1034] running on GCE node
I20260808 07:10:29.916407 4018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:29.916648 4018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:29.917832 4018 hybrid_clock.cc:648] HybridClock initialized: now 1786173029917819 us; error 44 us; skew 500 ppm
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:29.920290 4018 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:10:29.920768 4018 webserver.cc:460] Webserver started at http://127.30.54.67:39677/ using document root <none> and password file <none>
I20260808 07:10:29.920984 4018 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:29.921027 4018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:29.921180 4018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:29.922269 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "11d3d52712ae426aa609a7f7174b0603"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "176cc5d46f3d0422407b7880e4b488bd"
server_key_iv: "172411a77b0eb98644747116da16f131"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.922632 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "11d3d52712ae426aa609a7f7174b0603"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "176cc5d46f3d0422407b7880e4b488bd"
server_key_iv: "172411a77b0eb98644747116da16f131"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.924016 4018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:29.924751 4038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.924942 4018 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:29.925043 4018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "11d3d52712ae426aa609a7f7174b0603"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-znh6"
server_key: "176cc5d46f3d0422407b7880e4b488bd"
server_key_iv: "172411a77b0eb98644747116da16f131"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.925129 4018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:29.947548 4018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:29.949430 4018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:29.949612 4018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:29.949841 4018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:29.950155 4018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:29.950210 4018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.950251 4018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:29.950281 4018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.956075 4018 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:38253
I20260808 07:10:29.956127 4209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:38253 every 8 connection(s)
I20260808 07:10:29.956420 4018 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:29.962227 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 4018
I20260808 07:10:29.962339 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:29.962536 30937 external_mini_cluster.cc:1428] Setting key 3d46effe45172e086a5152aace9ea297
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:29.966097 3606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.957113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:37313 (local address 127.30.54.126:41991)
0808 07:10:29.957158 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.957161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.957369 (+ 208us) server_negotiation.cc:407] Connection header received
0808 07:10:29.958319 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.958321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.958333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.958401 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.959020 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.959642 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.960804 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.961011 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.961088 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.963636 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.963637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.963637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.963643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.964752 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.965103 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.965104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.965104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.965163 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.965558 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.965560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.965560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.965672 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.965737 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.965974 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.966036 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:10:29.966256 4213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.957068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41991 (local address 127.30.54.67:37313)
0808 07:10:29.957213 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:29.957256 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:29.958203 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:29.958470 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:29.958474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:29.958726 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:29.958922 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.958927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.959732 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.959733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.960716 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.960718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.961093 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.961094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.961255 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.961727 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:29.961735 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:29.963556 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:29.964818 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.964823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.964825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.965037 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.965222 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.965223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.965223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.965400 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.965741 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:29.965745 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:29.965850 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 07:10:29.965932 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:10:29.966047 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260808 07:10:29.966730 4211 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41991
I20260808 07:10:29.966849 4211 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:29.967029 4211 heartbeater.cc:511] Master 127.30.54.126:41991 requested a full tablet report, sending...
I20260808 07:10:29.967409 3557 ts_manager.cc:194] Registered new tserver with Master: 11d3d52712ae426aa609a7f7174b0603 (127.30.54.67:38253)
I20260808 07:10:29.967775 3557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:37313
I20260808 07:10:29.974452 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:29.997947 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.990317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34758 (local address 127.30.54.65:45283)
0808 07:10:29.990527 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.990532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.990543 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:29.990578 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.990579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.990592 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.990670 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.991320 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.992025 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.992767 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.993085 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.993168 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.995352 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.995361 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.995361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.995368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.996762 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.997123 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.997126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.997126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.997186 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.997384 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.997385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.997385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.997518 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.997707 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.997750 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.997830 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
Aug 08 07:10:29 dist-test-slave-znh6 krb5kdc[3495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:30.005507 3606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.999148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.30.54.126:41991)
0808 07:10:29.999209 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.999211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.999219 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:10:29.999281 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.999282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.999290 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.999335 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.999725 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.000186 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.000806 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.000985 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.001034 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.003070 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:30.003071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:30.003071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:30.003077 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:30.004450 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:30.004743 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:30.004744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:30.004744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:30.004792 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:30.005020 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:30.005021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:30.005021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:30.005151 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:30.005223 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.005390 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.005447 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 07:10:30.005780 3557 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59578
I20260808 07:10:30.006101 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 3637
I20260808 07:10:30.020761 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 3824
I20260808 07:10:30.027230 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 4018
I20260808 07:10:30.033758 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 3512
2026-08-08T07:10:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2032 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:30 dist-test-slave-znh6 krb5kdc[4229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:30 dist-test-slave-znh6 krb5kdc[4229](info): set up 2 sockets
Aug 08 07:10:30 dist-test-slave-znh6 krb5kdc[4229](info): commencing operation
krb5kdc: starting...
W20260808 07:10:31.244104 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:31 dist-test-slave-znh6 krb5kdc[4229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:31.366427 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41705
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:41705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:31.447631 4245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:31.447818 4245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:31.447841 4245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:31.448892 4245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:31.448925 4245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:31.448938 4245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:31.448951 4245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:31.448962 4245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:31.450332 4245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:41705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:31.450519 4245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:31.450680 4245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:31.455528 4256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:31.455627 4245 server_base.cc:1034] running on GCE node
W20260808 07:10:31.455559 4259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:31.455588 4255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:31.455993 4245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:31.456239 4245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:31.457381 4245 hybrid_clock.cc:648] HybridClock initialized: now 1786173031457359 us; error 41 us; skew 500 ppm
Aug 08 07:10:31 dist-test-slave-znh6 krb5kdc[4229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173031, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:31.463604 4245 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:31.463968 4245 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:31.464040 4245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:31.464192 4245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:31.465214 4245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "023f1f168148f9ba118385209ad81b3a"
server_key_iv: "2241a94fbdcfb9d69b9097c8d50ef0ee"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.465580 4245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "023f1f168148f9ba118385209ad81b3a"
server_key_iv: "2241a94fbdcfb9d69b9097c8d50ef0ee"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.466907 4245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:31.467648 4269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.467820 4245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:31.467918 4245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "023f1f168148f9ba118385209ad81b3a"
server_key_iv: "2241a94fbdcfb9d69b9097c8d50ef0ee"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.468001 4245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:31.476472 4245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:31.478200 4245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:31.478359 4245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:31.482551 4245 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:41705
I20260808 07:10:31.482599 4342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:41705 every 8 connection(s)
I20260808 07:10:31.482940 4245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:31.483860 4344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:31.486479 4344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1: Bootstrap starting.
I20260808 07:10:31.487092 4344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:31.487337 4344 log.cc:824] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:31.487931 4344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1: No bootstrap required, opened a new log
I20260808 07:10:31.489411 4344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41705 } }
I20260808 07:10:31.489543 4344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:31.489565 4344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb7d4a5d3f14f9195838ccc5e92c3b1, State: Initialized, Role: FOLLOWER
I20260808 07:10:31.489660 4344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41705 } }
I20260808 07:10:31.489709 4344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:31.489729 4344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:31.489797 4344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:31.490584 4344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41705 } }
I20260808 07:10:31.490700 4344 leader_election.cc:302] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6eb7d4a5d3f14f9195838ccc5e92c3b1; no voters:
I20260808 07:10:31.490908 4344 leader_election.cc:288] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:31.490986 4349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:31.491158 4349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [term 1 LEADER]: Becoming Leader. State: Replica: 6eb7d4a5d3f14f9195838ccc5e92c3b1, State: Running, Role: LEADER
I20260808 07:10:31.491191 4344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:31.491254 4349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41705 } }
I20260808 07:10:31.491690 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 4245
I20260808 07:10:31.491789 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:31.491997 30937 external_mini_cluster.cc:1428] Setting key 2815353cab62d3903ba9af0ab0f23110
I20260808 07:10:31.492003 4351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eb7d4a5d3f14f9195838ccc5e92c3b1. Latest consensus state: current_term: 1 leader_uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41705 } } }
I20260808 07:10:31.492075 4351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:31.492290 4350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb7d4a5d3f14f9195838ccc5e92c3b1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41705 } } }
I20260808 07:10:31.492364 4350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1 [sys.catalog]: This master's current role is: LEADER
W20260808 07:10:31.493068 4376 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:10:31.493136 4376 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:10:31.493185 4366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:31.493366 4366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:31.494745 4366 catalog_manager.cc:1347] Generated new cluster ID: 05087db673434be2beec8f0b221217f1
I20260808 07:10:31.494796 4366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:31 dist-test-slave-znh6 krb5kdc[4229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:31.503671 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:31.492568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43908 (local address 127.30.54.126:41705)
0808 07:10:31.492771 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:31.492782 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:31.492816 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:10:31.492961 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:31.492969 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:31.493026 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:31.493968 (+ 942us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:31.494060 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.495495 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.496552 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.497202 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.497316 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:31.500120 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:31.500124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:31.500124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:31.500137 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:31.502005 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:31.502413 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:31.502416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:31.502416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:31.502521 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:31.502738 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:31.502739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:31.502739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:31.503064 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:31.503200 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:31.503334 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 07:10:31.503469 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":73,"threads_started":1}
I20260808 07:10:31.513756 4366 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:31.514277 4366 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:31.519585 4366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1: Generated new TSK 0
I20260808 07:10:31.519768 4366 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:10:31.550288 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41705
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:31.631315 4385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:31.631511 4385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:31.631536 4385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:31.632648 4385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:31.632690 4385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:31.632745 4385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:31.634224 4385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:31.634460 4385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:31.634665 4385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:31.639564 4392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:31.639572 4395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:31.639572 4391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:31.639734 4385 server_base.cc:1034] running on GCE node
I20260808 07:10:31.639976 4385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:31.640178 4385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:31.641357 4385 hybrid_clock.cc:648] HybridClock initialized: now 1786173031641342 us; error 27 us; skew 500 ppm
Aug 08 07:10:31 dist-test-slave-znh6 krb5kdc[4229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173031, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:31.643841 4385 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:10:31.644184 4385 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:31.644263 4385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:31.644420 4385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:31.645551 4385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "cf7b7f85cf284b81bb79500b9927ecb5"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "ff1e2e53b3d48bdefe7727d7b92b37ce"
server_key_iv: "129bc9c9cc42874bdb2c72fc6f76f501"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.645922 4385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "cf7b7f85cf284b81bb79500b9927ecb5"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "ff1e2e53b3d48bdefe7727d7b92b37ce"
server_key_iv: "129bc9c9cc42874bdb2c72fc6f76f501"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.647358 4385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:31.648146 4401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.648341 4385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:31.648447 4385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "cf7b7f85cf284b81bb79500b9927ecb5"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "ff1e2e53b3d48bdefe7727d7b92b37ce"
server_key_iv: "129bc9c9cc42874bdb2c72fc6f76f501"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.648533 4385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:31.666311 4385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:31.668224 4385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:31.668406 4385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:31.668653 4385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:31.668984 4385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:31.669041 4385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.669083 4385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:31.669111 4385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.674934 4385 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:42019
I20260808 07:10:31.674993 4562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:42019 every 8 connection(s)
I20260808 07:10:31.675290 4385 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:10:31 dist-test-slave-znh6 krb5kdc[4229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173031, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:31.684275 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:31.675993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:58441 (local address 127.30.54.126:41705)
0808 07:10:31.676062 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:31.676066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:31.676301 (+ 235us) server_negotiation.cc:407] Connection header received
0808 07:10:31.676825 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:31.676826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:31.676838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:31.676887 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:31.677839 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.678356 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.679426 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.679649 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.679760 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:31.681744 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:31.681746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:31.681746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:31.681754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:31.683049 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:31.683424 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:31.683425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:31.683425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:31.683476 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:31.683680 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:31.683680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:31.683681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:31.683777 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:31.683826 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:31.684110 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 07:10:31.684199 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:10:31.684283 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:31.675982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41705 (local address 127.30.54.65:58441)
0808 07:10:31.676149 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:31.676185 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:31.676685 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:31.676979 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:31.676983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:31.677225 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:31.677714 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:31.677719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.678459 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.678460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:31.679332 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:31.679335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.679777 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.679779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:31.679901 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:31.680312 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:31.680320 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:31.681642 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:31.683127 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:31.683133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:31.683134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:31.683357 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:31.683533 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:31.683534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:31.683534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:31.683627 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:31.683831 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:31.683835 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:31.683937 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 07:10:31.684023 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:10:31.684133 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260808 07:10:31.684891 4563 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41705
I20260808 07:10:31.684974 4563 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:31.685081 4563 heartbeater.cc:511] Master 127.30.54.126:41705 requested a full tablet report, sending...
I20260808 07:10:31.685335 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 4385
I20260808 07:10:31.685464 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:31.685674 30937 external_mini_cluster.cc:1428] Setting key d534047999fea1f4d45d0dfd93011de4
I20260808 07:10:31.685765 4293 ts_manager.cc:194] Registered new tserver with Master: cf7b7f85cf284b81bb79500b9927ecb5 (127.30.54.65:42019)
I20260808 07:10:31.686959 4293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:58441
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:10:31.714474 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41705
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:31.794787 4568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:31.795032 4568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:31.795069 4568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:31.796185 4568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:31.796252 4568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:31.796296 4568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:31.797792 4568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:31.798050 4568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:31.798270 4568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:31.803143 4576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:31.803161 4578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:31.803148 4575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:31.803449 4568 server_base.cc:1034] running on GCE node
I20260808 07:10:31.803687 4568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:31.803913 4568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:31.805090 4568 hybrid_clock.cc:648] HybridClock initialized: now 1786173031805078 us; error 27 us; skew 500 ppm
Aug 08 07:10:31 dist-test-slave-znh6 krb5kdc[4229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173031, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:31.807516 4568 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:10:31.807799 4568 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:31.807842 4568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:31.807928 4568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:31.808849 4568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "d19856ee761049a080ead6b27745dc8e"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "79d49acf723acc8b25bb7e35133111a8"
server_key_iv: "5270803bdca19bb05f96105c7c75437f"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.809190 4568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "d19856ee761049a080ead6b27745dc8e"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "79d49acf723acc8b25bb7e35133111a8"
server_key_iv: "5270803bdca19bb05f96105c7c75437f"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.810443 4568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:10:31.811167 4584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.811352 4568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.811437 4568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "d19856ee761049a080ead6b27745dc8e"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "79d49acf723acc8b25bb7e35133111a8"
server_key_iv: "5270803bdca19bb05f96105c7c75437f"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.811514 4568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:31.827723 4568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:31.829423 4568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:31.829586 4568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:31.829800 4568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:31.830102 4568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:31.830137 4568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.830156 4568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:31.830204 4568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.836068 4568 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:34383
I20260808 07:10:31.836136 4744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:34383 every 8 connection(s)
I20260808 07:10:31.836376 4568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:31.839501 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 4568
I20260808 07:10:31.839581 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:31.839761 30937 external_mini_cluster.cc:1428] Setting key 53feb0e55810e6a10f91541f391b3b82
Aug 08 07:10:31 dist-test-slave-znh6 krb5kdc[4229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173031, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:31.846004 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:31.837057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:38647 (local address 127.30.54.126:41705)
0808 07:10:31.837129 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:31.837130 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:31.837332 (+ 202us) server_negotiation.cc:407] Connection header received
0808 07:10:31.837829 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:31.837829 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:31.837839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:31.837893 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:31.838699 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.839260 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.840483 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.840791 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.840867 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:31.843036 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:31.843038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:31.843039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:31.843046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:31.844630 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:31.845039 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:31.845041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:31.845041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:31.845105 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:31.845361 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:31.845362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:31.845362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:31.845486 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:31.845565 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:31.845825 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 07:10:31.845916 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:10:31.846182 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:31.837018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41705 (local address 127.30.54.66:38647)
0808 07:10:31.837202 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:31.837241 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:31.837743 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:31.837944 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:31.837947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:31.838179 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:31.838626 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:31.838630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.839337 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.839337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:31.840347 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:31.840367 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.840874 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.840875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:31.840984 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:31.841477 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:31.841490 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:31.842941 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:31.844725 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:31.844731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:31.844732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:31.844934 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:31.845178 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:31.845179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:31.845180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:31.845282 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:31.845555 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:31.845561 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:31.845686 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 07:10:31.845789 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 07:10:31.845940 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260808 07:10:31.846589 4745 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41705
I20260808 07:10:31.846678 4745 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:31.846858 4745 heartbeater.cc:511] Master 127.30.54.126:41705 requested a full tablet report, sending...
I20260808 07:10:31.847244 4293 ts_manager.cc:194] Registered new tserver with Master: d19856ee761049a080ead6b27745dc8e (127.30.54.66:34383)
I20260808 07:10:31.847628 4293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:38647
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:10:31.866441 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41705
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:31.946882 4757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:31.947069 4757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:31.947093 4757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:31.948173 4757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:31.948208 4757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:31.948235 4757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:31.949626 4757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:31.949792 4757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:31.949952 4757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:31.954706 4769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:31.954715 4766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:31.954985 4757 server_base.cc:1034] running on GCE node
W20260808 07:10:31.955045 4762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:31.955238 4757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:31.955418 4757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:31.956588 4757 hybrid_clock.cc:648] HybridClock initialized: now 1786173031956577 us; error 40 us; skew 500 ppm
Aug 08 07:10:31 dist-test-slave-znh6 krb5kdc[4229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173031, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:31.959241 4757 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:10:31.959566 4757 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:31.959640 4757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:31.959787 4757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:31.960763 4757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "076381efeecd4a0aaa1a185ef428027c"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "e0a3a160e9ca6df6eae43d64f561f612"
server_key_iv: "f3a94e0ea602f5589fc70833eba49977"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.961120 4757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "076381efeecd4a0aaa1a185ef428027c"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "e0a3a160e9ca6df6eae43d64f561f612"
server_key_iv: "f3a94e0ea602f5589fc70833eba49977"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.962366 4757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:10:31.963101 4777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.963279 4757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:31.963382 4757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "076381efeecd4a0aaa1a185ef428027c"
format_stamp: "Formatted at 2026-08-08 07:10:31 on dist-test-slave-znh6"
server_key: "e0a3a160e9ca6df6eae43d64f561f612"
server_key_iv: "f3a94e0ea602f5589fc70833eba49977"
server_key_version: "encryptionkey@0"
I20260808 07:10:31.963454 4757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:31.975495 4757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:31.977692 4757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:31.977844 4757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:31.978067 4757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:31.978364 4757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:31.978418 4757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.978457 4757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:31.978485 4757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:31.984460 4757 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:34761
I20260808 07:10:31.984494 4935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:34761 every 8 connection(s)
I20260808 07:10:31.984759 4757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:10:31 dist-test-slave-znh6 krb5kdc[4229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173031, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:31.991196 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 4757
I20260808 07:10:31.991281 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:31.991503 30937 external_mini_cluster.cc:1428] Setting key ca898b4ac3e047dcc0ce174edf4bdc38
I20260808 07:10:31.993902 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:31.985453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:57069 (local address 127.30.54.126:41705)
0808 07:10:31.985515 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:31.985517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:31.985723 (+ 206us) server_negotiation.cc:407] Connection header received
0808 07:10:31.986155 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:31.986156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:31.986165 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:31.986212 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:31.987133 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.987610 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.988522 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.988711 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.988764 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:31.990645 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:31.990646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:31.990646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:31.990653 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:31.992318 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:31.992726 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:31.992727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:31.992727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:31.992827 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:31.993162 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:31.993163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:31.993164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:31.993318 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:31.993398 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:31.993702 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 07:10:31.993794 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:10:31.994129 4939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:31.985434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41705 (local address 127.30.54.67:57069)
0808 07:10:31.985611 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:31.985644 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:31.986075 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:31.986266 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:31.986269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:31.986506 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:31.987058 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:31.987063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.987684 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.987685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:31.988449 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:31.988451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.988784 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.988785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:31.988878 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:31.989270 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:31.989277 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:31.990564 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:31.992390 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:31.992397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:31.992399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:31.992655 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:31.992899 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:31.992900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:31.992900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:31.993085 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:31.993393 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:31.993399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:31.993544 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 07:10:31.993670 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 07:10:31.993834 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260808 07:10:31.994594 4936 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41705
I20260808 07:10:31.994696 4936 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:31.994879 4936 heartbeater.cc:511] Master 127.30.54.126:41705 requested a full tablet report, sending...
I20260808 07:10:31.995246 4293 ts_manager.cc:194] Registered new tserver with Master: 076381efeecd4a0aaa1a185ef428027c (127.30.54.67:34761)
I20260808 07:10:31.995616 4293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:57069
I20260808 07:10:32.003295 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:32.010288 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.004538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43920 (local address 127.30.54.126:41705)
0808 07:10:32.004596 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.004598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.004695 (+ 97us) server_negotiation.cc:407] Connection header received
0808 07:10:32.004771 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.004772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.004782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.004852 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.005391 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.005854 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.006574 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.006792 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.006930 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.008023 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.008024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.008025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:32.008032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.009139 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:32.009499 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:32.009500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:32.009501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:32.009561 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:32.009757 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:32.009758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:32.009758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:32.009901 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.010015 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.010112 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.010189 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:10:32.011729 4293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:32.013293 4293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:32.019796 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.015564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:42019 (local address 127.0.0.1:41592)
0808 07:10:32.015968 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.015978 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.016023 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.016400 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.016401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.016402 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:32.016438 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.016438 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.017335 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.017336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.018571 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.018574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.019512 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.019513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.019590 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.019605 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.019671 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.019725 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":47,"threads_started":1}
I20260808 07:10:32.019927 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.015528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:34761 (local address 127.0.0.1:58500)
0808 07:10:32.015646 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.015685 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.015750 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.016172 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.016176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.016179 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:32.016250 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.016253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.017708 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.017709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.018740 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.018742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.019620 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.019621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.019706 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.019711 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.019808 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.019853 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":74,"threads_started":1}
I20260808 07:10:32.019940 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.015781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58500 (local address 127.30.54.67:34761)
0808 07:10:32.015980 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.015985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.015996 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:32.016037 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.016039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.016055 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.016346 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:32.016424 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.017597 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.018842 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.019516 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.019613 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.019663 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.019827 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.019864 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260808 07:10:32.020136 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.015805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41592 (local address 127.30.54.65:42019)
0808 07:10:32.016144 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.016148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.016159 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:32.016199 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.016201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.016216 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.016289 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:32.016522 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.017232 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.018660 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.019421 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.019502 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.019566 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.019993 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.020037 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260808 07:10:32.020733 4463 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed69393c6df4cda9d01a04cd5651ac3 (DEFAULT_TABLE table=test-table [id=699053a77d914fca9e3b70b8d0c33d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:32.020784 4842 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed69393c6df4cda9d01a04cd5651ac3 (DEFAULT_TABLE table=test-table [id=699053a77d914fca9e3b70b8d0c33d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:32.021070 4463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed69393c6df4cda9d01a04cd5651ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:32.021137 4842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed69393c6df4cda9d01a04cd5651ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:32.021389 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.015535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:34383 (local address 127.0.0.1:38716)
0808 07:10:32.015775 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.015785 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.015840 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.016389 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.016390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.016393 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:32.016436 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.016438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.018808 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.018808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.019576 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.019578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.021172 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.021173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.021239 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.021242 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.021275 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.021314 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":58,"threads_started":1}
I20260808 07:10:32.021787 4953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.016032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.30.54.66:34383)
0808 07:10:32.016225 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.016229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.016239 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:32.016274 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.016277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.016293 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.016377 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:32.016500 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.017582 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.019676 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.021066 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.021579 (+ 513us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.021641 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.021667 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.021704 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":83,"threads_started":1}
I20260808 07:10:32.022347 4656 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed69393c6df4cda9d01a04cd5651ac3 (DEFAULT_TABLE table=test-table [id=699053a77d914fca9e3b70b8d0c33d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:32.022684 4656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed69393c6df4cda9d01a04cd5651ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:32.024096 4954 tablet_bootstrap.cc:492] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5: Bootstrap starting.
I20260808 07:10:32.024489 4955 tablet_bootstrap.cc:492] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: Bootstrap starting.
I20260808 07:10:32.024868 4954 tablet_bootstrap.cc:654] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:32.025116 4955 tablet_bootstrap.cc:654] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:32.025451 4954 log.cc:824] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:32.025698 4955 log.cc:824] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:32.026031 4960 tablet_bootstrap.cc:492] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: Bootstrap starting.
I20260808 07:10:32.026369 4954 tablet_bootstrap.cc:492] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5: No bootstrap required, opened a new log
I20260808 07:10:32.026453 4954 ts_tablet_manager.cc:1397] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:32.026607 4960 tablet_bootstrap.cc:654] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:32.027344 4955 tablet_bootstrap.cc:492] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: No bootstrap required, opened a new log
I20260808 07:10:32.027395 4955 ts_tablet_manager.cc:1397] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:10:32.028013 4954 raft_consensus.cc:348] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } }
I20260808 07:10:32.028165 4954 raft_consensus.cc:374] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:32.028213 4954 raft_consensus.cc:729] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf7b7f85cf284b81bb79500b9927ecb5, State: Initialized, Role: FOLLOWER
I20260808 07:10:32.028338 4954 consensus_queue.cc:260] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } }
I20260808 07:10:32.028611 4563 heartbeater.cc:503] Master 127.30.54.126:41705 was elected leader, sending a full tablet report...
I20260808 07:10:32.028618 4954 ts_tablet_manager.cc:1428] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:32.028826 4955 raft_consensus.cc:348] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } }
I20260808 07:10:32.029018 4955 raft_consensus.cc:374] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:32.029057 4955 raft_consensus.cc:729] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076381efeecd4a0aaa1a185ef428027c, State: Initialized, Role: FOLLOWER
I20260808 07:10:32.029199 4955 consensus_queue.cc:260] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } }
I20260808 07:10:32.029505 4955 ts_tablet_manager.cc:1428] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:32.029520 4936 heartbeater.cc:503] Master 127.30.54.126:41705 was elected leader, sending a full tablet report...
I20260808 07:10:32.029821 4960 log.cc:824] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:32.030651 4960 tablet_bootstrap.cc:492] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: No bootstrap required, opened a new log
I20260808 07:10:32.030705 4960 ts_tablet_manager.cc:1397] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 07:10:32.032019 4960 raft_consensus.cc:348] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } }
I20260808 07:10:32.032135 4960 raft_consensus.cc:374] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:32.032155 4960 raft_consensus.cc:729] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d19856ee761049a080ead6b27745dc8e, State: Initialized, Role: FOLLOWER
I20260808 07:10:32.032253 4960 consensus_queue.cc:260] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } }
I20260808 07:10:32.032473 4960 ts_tablet_manager.cc:1428] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:32.032549 4745 heartbeater.cc:503] Master 127.30.54.126:41705 was elected leader, sending a full tablet report...
W20260808 07:10:32.087049 4747 tablet.cc:2369] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:32.176009 4564 tablet.cc:2369] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:32.186928 4965 raft_consensus.cc:482] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:32.187042 4965 raft_consensus.cc:504] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } }
I20260808 07:10:32.187731 4965 leader_election.cc:288] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 076381efeecd4a0aaa1a185ef428027c (127.30.54.67:34761), cf7b7f85cf284b81bb79500b9927ecb5 (127.30.54.65:42019)
I20260808 07:10:32.191717 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.188031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:33471 (local address 127.30.54.65:42019)
0808 07:10:32.188072 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.188074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.188168 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:10:32.188229 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.188230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.188238 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.188282 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:32.188446 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.189193 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.190763 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.191325 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.191421 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.191432 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.191610 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.191650 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:32.191717 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.187982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:56779 (local address 127.30.54.67:34761)
0808 07:10:32.188032 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.188034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.188045 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:32.188076 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.188077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.188087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.188170 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:32.188384 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.189178 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.190760 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.191391 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.191463 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.191470 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.191609 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.191648 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:10:32.191732 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.187886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:34761 (local address 127.30.54.66:56779)
0808 07:10:32.187931 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.187945 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.187998 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.188188 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.188190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.188191 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:32.188304 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.188306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.189297 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.189297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.190666 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.190669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.191456 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.191457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.191537 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.191540 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.191594 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.191639 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":69}
I20260808 07:10:32.191740 4966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.187921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:42019 (local address 127.30.54.66:33471)
0808 07:10:32.188094 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.188101 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.188171 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.188302 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.188303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.188305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:32.188370 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.188371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.189284 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.189285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.190666 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:32.190669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.191432 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.191433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:32.191513 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.191519 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.191587 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.191639 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":76,"thread_start_us":101,"threads_started":1}
I20260808 07:10:32.192098 4871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed69393c6df4cda9d01a04cd5651ac3" candidate_uuid: "d19856ee761049a080ead6b27745dc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076381efeecd4a0aaa1a185ef428027c" is_pre_election: true
I20260808 07:10:32.192276 4871 raft_consensus.cc:2454] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d19856ee761049a080ead6b27745dc8e in term 0.
I20260808 07:10:32.192308 4490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed69393c6df4cda9d01a04cd5651ac3" candidate_uuid: "d19856ee761049a080ead6b27745dc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" is_pre_election: true
I20260808 07:10:32.192498 4490 raft_consensus.cc:2454] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d19856ee761049a080ead6b27745dc8e in term 0.
I20260808 07:10:32.192500 4592 leader_election.cc:302] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 076381efeecd4a0aaa1a185ef428027c, d19856ee761049a080ead6b27745dc8e; no voters:
I20260808 07:10:32.192658 4965 raft_consensus.cc:2781] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:32.192696 4965 raft_consensus.cc:482] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:32.192713 4965 raft_consensus.cc:3037] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:32.193399 4965 raft_consensus.cc:504] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } }
I20260808 07:10:32.193594 4965 leader_election.cc:288] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [CANDIDATE]: Term 1 election: Requested vote from peers 076381efeecd4a0aaa1a185ef428027c (127.30.54.67:34761), cf7b7f85cf284b81bb79500b9927ecb5 (127.30.54.65:42019)
I20260808 07:10:32.193696 4490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed69393c6df4cda9d01a04cd5651ac3" candidate_uuid: "d19856ee761049a080ead6b27745dc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7b7f85cf284b81bb79500b9927ecb5"
I20260808 07:10:32.193739 4871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed69393c6df4cda9d01a04cd5651ac3" candidate_uuid: "d19856ee761049a080ead6b27745dc8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076381efeecd4a0aaa1a185ef428027c"
I20260808 07:10:32.193787 4490 raft_consensus.cc:3037] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:32.193813 4871 raft_consensus.cc:3037] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:32.194496 4871 raft_consensus.cc:2454] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d19856ee761049a080ead6b27745dc8e in term 1.
I20260808 07:10:32.194540 4490 raft_consensus.cc:2454] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d19856ee761049a080ead6b27745dc8e in term 1.
I20260808 07:10:32.194694 4592 leader_election.cc:302] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 076381efeecd4a0aaa1a185ef428027c, d19856ee761049a080ead6b27745dc8e; no voters:
I20260808 07:10:32.194784 4965 raft_consensus.cc:2781] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:32.194970 4965 raft_consensus.cc:686] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 1 LEADER]: Becoming Leader. State: Replica: d19856ee761049a080ead6b27745dc8e, State: Running, Role: LEADER
I20260808 07:10:32.195066 4965 consensus_queue.cc:237] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } }
I20260808 07:10:32.195822 4293 catalog_manager.cc:5495] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e reported cstate change: term changed from 0 to 1, leader changed from <none> to d19856ee761049a080ead6b27745dc8e (127.30.54.66). New cstate: current_term: 1 leader_uuid: "d19856ee761049a080ead6b27745dc8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d19856ee761049a080ead6b27745dc8e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 34383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:32.201335 4953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.199296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38718 (local address 127.30.54.66:34383)
0808 07:10:32.199337 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.199338 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.199359 (+ 21us) server_negotiation.cc:407] Connection header received
0808 07:10:32.199432 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.199432 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.199446 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.199509 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:32.199596 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.200022 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.200735 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.200904 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.200962 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.201099 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:32.201180 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:32.201239 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.201273 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.201289 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:10:32.202951 4871 raft_consensus.cc:1261] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [term 1 FOLLOWER]: Refusing update from remote peer d19856ee761049a080ead6b27745dc8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:32.202958 4490 raft_consensus.cc:1261] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [term 1 FOLLOWER]: Refusing update from remote peer d19856ee761049a080ead6b27745dc8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:32.203298 4967 consensus_queue.cc:1035] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "076381efeecd4a0aaa1a185ef428027c" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:32.203435 4965 consensus_queue.cc:1035] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf7b7f85cf284b81bb79500b9927ecb5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 42019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:32.205756 4973 mvcc.cc:205] Tried to move back new op lower bound from 7316164739900547072 to 7316164739871485952. Current Snapshot: MvccSnapshot[applied={T|T < 7316164739900547072}]
I20260808 07:10:32.207845 4289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:32.207945 4289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:32.208621 4289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6eb7d4a5d3f14f9195838ccc5e92c3b1: Sending DeleteTablet for 3 replicas of tablet 2ed69393c6df4cda9d01a04cd5651ac3
I20260808 07:10:32.208976 4463 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ed69393c6df4cda9d01a04cd5651ac3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:32 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:41592
I20260808 07:10:32.209004 4842 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ed69393c6df4cda9d01a04cd5651ac3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:32 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:58500
I20260808 07:10:32.209190 4985 tablet_replica.cc:330] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5: stopping tablet replica
I20260808 07:10:32.209228 4986 tablet_replica.cc:330] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: stopping tablet replica
I20260808 07:10:32.209219 4656 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ed69393c6df4cda9d01a04cd5651ac3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:32 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:38716
I20260808 07:10:32.209280 4985 raft_consensus.cc:2229] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:32.209303 4986 raft_consensus.cc:2229] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:32.209381 4985 raft_consensus.cc:2258] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:32.209381 4986 raft_consensus.cc:2258] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:32.209805 4987 tablet_replica.cc:330] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: stopping tablet replica
I20260808 07:10:32.209827 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 4385
I20260808 07:10:32.209824 4985 ts_tablet_manager.cc:1905] T 2ed69393c6df4cda9d01a04cd5651ac3 P cf7b7f85cf284b81bb79500b9927ecb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:32.212282 4987 raft_consensus.cc:2229] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:32.212406 4987 raft_consensus.cc:2258] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:32.214231 4987 ts_tablet_manager.cc:1905] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:32.216111 4987 ts_tablet_manager.cc:1918] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:32.216208 4987 log.cc:1196] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/2ed69393c6df4cda9d01a04cd5651ac3
I20260808 07:10:32.216516 4987 ts_tablet_manager.cc:1939] T 2ed69393c6df4cda9d01a04cd5651ac3 P d19856ee761049a080ead6b27745dc8e: Deleting consensus metadata
I20260808 07:10:32.216924 4272 catalog_manager.cc:4841] TS d19856ee761049a080ead6b27745dc8e (127.30.54.66:34383): tablet 2ed69393c6df4cda9d01a04cd5651ac3 (table test-table [id=699053a77d914fca9e3b70b8d0c33d0e]) successfully deleted
I20260808 07:10:32.228579 4986 ts_tablet_manager.cc:1905] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:32.230240 4986 ts_tablet_manager.cc:1918] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:32.230320 4986 log.cc:1196] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/2ed69393c6df4cda9d01a04cd5651ac3
I20260808 07:10:32.230592 4986 ts_tablet_manager.cc:1939] T 2ed69393c6df4cda9d01a04cd5651ac3 P 076381efeecd4a0aaa1a185ef428027c: Deleting consensus metadata
I20260808 07:10:32.231060 4274 catalog_manager.cc:4841] TS 076381efeecd4a0aaa1a185ef428027c (127.30.54.67:34761): tablet 2ed69393c6df4cda9d01a04cd5651ac3 (table test-table [id=699053a77d914fca9e3b70b8d0c33d0e]) successfully deleted
W20260808 07:10:32.232837 4273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:42019 (error 108)
I20260808 07:10:32.233142 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 4568
I20260808 07:10:32.233441 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.233286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:42019 (local address 127.0.0.1:41602)
0808 07:10:32.233336 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.233391 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:42019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 07:10:32.235585 4273 catalog_manager.cc:4584] TS cf7b7f85cf284b81bb79500b9927ecb5 (127.30.54.65:42019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2ed69393c6df4cda9d01a04cd5651ac3: Network error: Client connection negotiation failed: client connection to 127.30.54.65:42019: connect: Connection refused (error 111)
I20260808 07:10:32.242244 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 4757
I20260808 07:10:32.249298 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 4245
2026-08-08T07:10:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2214 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T07:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:32Z Disabled control of system clock
I20260808 07:10:32.279676 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39879
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:39879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:10:32.359664 4996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:32.359853 4996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:32.359874 4996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:32.360996 4996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:32.361035 4996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:32.361048 4996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:32.361063 4996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:32.361101 4996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:32.362411 4996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:39879
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:32.362649 4996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:32.362977 4996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:32.367736 5010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.367759 5006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.367770 5005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:32.367997 4996 server_base.cc:1034] running on GCE node
I20260808 07:10:32.368227 4996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:32.368472 4996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:32.369679 4996 hybrid_clock.cc:648] HybridClock initialized: now 1786173032369662 us; error 30 us; skew 500 ppm
I20260808 07:10:32.371096 4996 webserver.cc:460] Webserver started at http://127.30.54.126:35867/ using document root <none> and password file <none>
I20260808 07:10:32.371372 4996 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:32.371424 4996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:32.371584 4996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:32.372816 4996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "445b67b1098f42d1b3e2a8a3195a6ea8"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "61fb315407f57548e446028e5ca559d8"
server_key_iv: "5c203b6fd1cbcac924ac9691fe399402"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.373204 4996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "445b67b1098f42d1b3e2a8a3195a6ea8"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "61fb315407f57548e446028e5ca559d8"
server_key_iv: "5c203b6fd1cbcac924ac9691fe399402"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.374614 4996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:32.375490 5017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.375675 4996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:32.375768 4996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "445b67b1098f42d1b3e2a8a3195a6ea8"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "61fb315407f57548e446028e5ca559d8"
server_key_iv: "5c203b6fd1cbcac924ac9691fe399402"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.375850 4996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:32.379150 4996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:32.379408 4996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:32.379544 4996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:32.383333 4996 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:39879
I20260808 07:10:32.383424 5090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:39879 every 8 connection(s)
I20260808 07:10:32.383700 4996 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:32.384336 5092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:32.386974 5092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8: Bootstrap starting.
I20260808 07:10:32.387486 5092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:32.387717 5092 log.cc:824] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:32.388338 5092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8: No bootstrap required, opened a new log
I20260808 07:10:32.389876 5092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445b67b1098f42d1b3e2a8a3195a6ea8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39879 } }
I20260808 07:10:32.390007 5092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:32.390029 5092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 445b67b1098f42d1b3e2a8a3195a6ea8, State: Initialized, Role: FOLLOWER
I20260808 07:10:32.390158 5092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445b67b1098f42d1b3e2a8a3195a6ea8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39879 } }
I20260808 07:10:32.390232 5092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:32.390254 5092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:32.390280 5092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:32.390765 5092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445b67b1098f42d1b3e2a8a3195a6ea8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39879 } }
I20260808 07:10:32.390905 5092 leader_election.cc:302] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 445b67b1098f42d1b3e2a8a3195a6ea8; no voters:
I20260808 07:10:32.391036 5092 leader_election.cc:288] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:32.391121 5097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:32.391279 5097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [term 1 LEADER]: Becoming Leader. State: Replica: 445b67b1098f42d1b3e2a8a3195a6ea8, State: Running, Role: LEADER
I20260808 07:10:32.391330 5092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:32.391404 5097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445b67b1098f42d1b3e2a8a3195a6ea8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39879 } }
I20260808 07:10:32.391786 5101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 445b67b1098f42d1b3e2a8a3195a6ea8. Latest consensus state: current_term: 1 leader_uuid: "445b67b1098f42d1b3e2a8a3195a6ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445b67b1098f42d1b3e2a8a3195a6ea8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39879 } } }
I20260808 07:10:32.391875 5101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:32.392058 5100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "445b67b1098f42d1b3e2a8a3195a6ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "445b67b1098f42d1b3e2a8a3195a6ea8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39879 } } }
I20260808 07:10:32.392112 5108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:32.392164 5100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:32.392853 5108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:32.393756 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 4996
I20260808 07:10:32.393855 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:32.394037 30937 external_mini_cluster.cc:1428] Setting key 4bd11b7e2ddf5f62ce6c28a4768f73f2
I20260808 07:10:32.394625 5108 catalog_manager.cc:1347] Generated new cluster ID: 1c365160833f4296b93ed6b483c347fd
I20260808 07:10:32.394677 5108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:32.396802 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.394506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.30.54.126:39879)
0808 07:10:32.394933 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.394941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.394972 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:10:32.395076 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.395082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.395134 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.395243 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.396357 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.396361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.396361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.396365 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.396415 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.396428 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.396492 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.396583 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.396635 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":82,"threads_started":1}
I20260808 07:10:32.410089 5108 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:32.410257 5108 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:32.415830 5108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8: Generated new TSK 0
I20260808 07:10:32.416020 5108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:32.421932 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39879
--builtin_ntp_servers=127.30.54.84:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:10:32.505185 5129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:32.505401 5129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:32.505439 5129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:32.506588 5129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:32.506649 5129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:32.506690 5129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:32.508044 5129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:32.508292 5129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:32.508577 5129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:32.514631 5140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.514629 5138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.514665 5136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:32.514899 5129 server_base.cc:1034] running on GCE node
I20260808 07:10:32.515136 5129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:32.515379 5129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:32.516618 5129 hybrid_clock.cc:648] HybridClock initialized: now 1786173032516599 us; error 33 us; skew 500 ppm
I20260808 07:10:32.517985 5129 webserver.cc:460] Webserver started at http://127.30.54.65:44627/ using document root <none> and password file <none>
I20260808 07:10:32.518262 5129 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:32.518334 5129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:32.518486 5129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:32.519609 5129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "a938a8fe3a824ef082fdeead750b61b7"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "210d46ca18b3615617cbf5a7a62352b9"
server_key_iv: "bfda51d98d6ff399e507a5322c3a8e3a"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.519979 5129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "a938a8fe3a824ef082fdeead750b61b7"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "210d46ca18b3615617cbf5a7a62352b9"
server_key_iv: "bfda51d98d6ff399e507a5322c3a8e3a"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.521306 5129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:10:32.522096 5153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.522305 5129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:32.522389 5129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "a938a8fe3a824ef082fdeead750b61b7"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "210d46ca18b3615617cbf5a7a62352b9"
server_key_iv: "bfda51d98d6ff399e507a5322c3a8e3a"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.522485 5129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:32.525733 5129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:32.525997 5129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:32.526145 5129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:32.526363 5129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:32.526676 5129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:32.526731 5129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.526768 5129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:32.526796 5129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.532231 5129 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:43733
I20260808 07:10:32.532303 5316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:43733 every 8 connection(s)
I20260808 07:10:32.532627 5129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:32.535681 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.533344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:37787 (local address 127.30.54.126:39879)
0808 07:10:32.533384 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.533386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.533610 (+ 224us) server_negotiation.cc:407] Connection header received
0808 07:10:32.534106 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.534107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.534115 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.534166 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.535391 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.535392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.535392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.535396 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.535419 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.535424 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.535498 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.535594 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.535609 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:10:32.535900 5319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.533309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39879 (local address 127.30.54.65:37787)
0808 07:10:32.533497 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.533527 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.533993 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.534230 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.534234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.534243 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:32.535234 (+ 991us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:32.535234 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:32.535261 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:32.535288 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:32.535497 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:32.535504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:32.535507 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.535605 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.535664 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260808 07:10:32.536405 5317 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39879
I20260808 07:10:32.536497 5317 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:32.536602 5317 heartbeater.cc:511] Master 127.30.54.126:39879 requested a full tablet report, sending...
I20260808 07:10:32.537074 5044 ts_manager.cc:194] Registered new tserver with Master: a938a8fe3a824ef082fdeead750b61b7 (127.30.54.65:43733)
I20260808 07:10:32.539747 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 5129
I20260808 07:10:32.539862 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:32.540036 30937 external_mini_cluster.cc:1428] Setting key 0b276ce032994b7c3de1df8d8c097893
I20260808 07:10:32.540920 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39879
--builtin_ntp_servers=127.30.54.84:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:10:32.621268 5320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:32.621502 5320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:32.621557 5320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:32.622745 5320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:32.622795 5320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:32.622859 5320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:32.624390 5320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:32.624617 5320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:32.624934 5320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:32.629963 5331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.630019 5329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.630021 5328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:32.630048 5320 server_base.cc:1034] running on GCE node
I20260808 07:10:32.630350 5320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:32.630609 5320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:32.631754 5320 hybrid_clock.cc:648] HybridClock initialized: now 1786173032631741 us; error 48 us; skew 500 ppm
I20260808 07:10:32.632941 5320 webserver.cc:460] Webserver started at http://127.30.54.66:37801/ using document root <none> and password file <none>
I20260808 07:10:32.633165 5320 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:32.633209 5320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:32.633351 5320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:32.634377 5320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "4a500a092fd340df8c74fa5ec3b719ea"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "5d1dab36d890d118968c8323db595cac"
server_key_iv: "e43f88b22ce5314a244f2912c5d2092a"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.634733 5320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "4a500a092fd340df8c74fa5ec3b719ea"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "5d1dab36d890d118968c8323db595cac"
server_key_iv: "e43f88b22ce5314a244f2912c5d2092a"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.636049 5320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:32.636747 5338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.637017 5320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:32.637112 5320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "4a500a092fd340df8c74fa5ec3b719ea"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "5d1dab36d890d118968c8323db595cac"
server_key_iv: "e43f88b22ce5314a244f2912c5d2092a"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.637167 5320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:32.640475 5320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:32.640671 5320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:32.640760 5320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:32.640918 5320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:32.641160 5320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:32.641191 5320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.641209 5320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:32.641222 5320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.646803 5320 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:40427
I20260808 07:10:32.646883 5490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:40427 every 8 connection(s)
I20260808 07:10:32.647205 5320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:32.649910 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.647964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:47323 (local address 127.30.54.126:39879)
0808 07:10:32.648016 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.648019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.648235 (+ 216us) server_negotiation.cc:407] Connection header received
0808 07:10:32.648639 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.648640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.648645 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.648699 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.649718 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.649719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.649720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.649723 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.649739 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.649742 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.649796 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.649841 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.649852 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:32.650089 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.647948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39879 (local address 127.30.54.66:47323)
0808 07:10:32.648124 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.648154 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.648559 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.648745 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.648749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.648756 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:32.649587 (+ 831us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:32.649587 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:32.649613 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:32.649639 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:32.649795 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:32.649797 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:32.649799 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.649847 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.649890 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260808 07:10:32.650612 5491 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39879
I20260808 07:10:32.650709 5491 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:32.650869 5491 heartbeater.cc:511] Master 127.30.54.126:39879 requested a full tablet report, sending...
I20260808 07:10:32.651225 5044 ts_manager.cc:194] Registered new tserver with Master: 4a500a092fd340df8c74fa5ec3b719ea (127.30.54.66:40427)
I20260808 07:10:32.655965 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 5320
I20260808 07:10:32.656054 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:32.656185 30937 external_mini_cluster.cc:1428] Setting key 7737811cf2bafb32bca6a909f1737686
I20260808 07:10:32.656976 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39879
--builtin_ntp_servers=127.30.54.84:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:10:32.737367 5495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:32.737551 5495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:32.737573 5495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:32.738629 5495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:32.738662 5495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:32.738686 5495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:32.740034 5495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:32.740201 5495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:32.740427 5495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:32.745400 5505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.745394 5503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.745393 5502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:32.745584 5495 server_base.cc:1034] running on GCE node
I20260808 07:10:32.745796 5495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:32.746030 5495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:32.747187 5495 hybrid_clock.cc:648] HybridClock initialized: now 1786173032747178 us; error 39 us; skew 500 ppm
I20260808 07:10:32.748363 5495 webserver.cc:460] Webserver started at http://127.30.54.67:38337/ using document root <none> and password file <none>
I20260808 07:10:32.748593 5495 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:32.748661 5495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:32.748798 5495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:32.749830 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "bfdca9f17db2467482f8095bf8c38594"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "4ce6a7629014cd329ff0fda252369171"
server_key_iv: "8feda67eeb98ba207ecf5b8dcd5ae28b"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.750185 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "bfdca9f17db2467482f8095bf8c38594"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "4ce6a7629014cd329ff0fda252369171"
server_key_iv: "8feda67eeb98ba207ecf5b8dcd5ae28b"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.751536 5495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:32.752319 5513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.752485 5495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:32.752585 5495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "bfdca9f17db2467482f8095bf8c38594"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-znh6"
server_key: "4ce6a7629014cd329ff0fda252369171"
server_key_iv: "8feda67eeb98ba207ecf5b8dcd5ae28b"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.752673 5495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:32.755954 5495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:32.756204 5495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:32.756337 5495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:32.756543 5495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:32.756831 5495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:32.756886 5495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.756924 5495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:32.756951 5495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.762661 5495 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:38059
I20260808 07:10:32.762697 5662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:38059 every 8 connection(s)
I20260808 07:10:32.763055 5495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:32.765861 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.763889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:59289 (local address 127.30.54.126:39879)
0808 07:10:32.763930 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.763932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.764128 (+ 196us) server_negotiation.cc:407] Connection header received
0808 07:10:32.764571 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.764572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.764587 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.764650 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.765672 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.765674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.765674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.765679 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.765695 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.765700 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.765747 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.765797 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.765809 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:10:32.766065 5665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.763856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39879 (local address 127.30.54.67:59289)
0808 07:10:32.764026 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.764056 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.764471 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.764697 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.764700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.764708 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:32.765536 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:32.765536 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:32.765560 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:32.765585 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:32.765750 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:32.765752 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:32.765754 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.765801 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.765844 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260808 07:10:32.766614 5663 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39879
I20260808 07:10:32.766726 5663 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:32.766883 5663 heartbeater.cc:511] Master 127.30.54.126:39879 requested a full tablet report, sending...
I20260808 07:10:32.767249 5044 ts_manager.cc:194] Registered new tserver with Master: bfdca9f17db2467482f8095bf8c38594 (127.30.54.67:38059)
I20260808 07:10:32.772138 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 5495
I20260808 07:10:32.772248 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:32.772423 30937 external_mini_cluster.cc:1428] Setting key 66cc8d48ba3ee718b5dad788781cbb5b
I20260808 07:10:32.773260 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:32.775712 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.774566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50492 (local address 127.30.54.126:39879)
0808 07:10:32.774645 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.774647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.774744 (+ 97us) server_negotiation.cc:407] Connection header received
0808 07:10:32.774848 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.774849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.774856 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.774905 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.775453 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.775454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.775455 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.775460 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.775475 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.775479 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.775554 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.775621 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.775639 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 07:10:32.777180 5044 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:32.778823 5044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:32.782577 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.780888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:43733 (local address 127.0.0.1:58966)
0808 07:10:32.781101 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.781137 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.781195 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.781593 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.781600 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.781607 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:32.782130 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:32.782131 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:32.782147 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:32.782170 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:32.782419 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:32.782424 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:32.782427 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.782489 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.782504 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":64,"threads_started":1}
I20260808 07:10:32.782752 5677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.781045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.30.54.67:38059)
0808 07:10:32.781211 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.781215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.781546 (+ 331us) server_negotiation.cc:407] Connection header received
0808 07:10:32.781831 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.781837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.781847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.781925 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.782369 (+ 444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.782375 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.782376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.782380 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.782412 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.782432 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.782489 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.782556 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.782570 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260808 07:10:32.782793 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.780954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:38059 (local address 127.0.0.1:43076)
0808 07:10:32.781464 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.781473 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.781547 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.781940 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.781941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.781944 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:32.782288 (+ 344us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:32.782288 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:32.782293 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:32.782299 (+ 6us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:32.782499 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:32.782500 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:32.782501 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.782551 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.782565 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":102,"threads_started":1}
I20260808 07:10:32.782925 5680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.781234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.30.54.65:43733)
0808 07:10:32.781403 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.781406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.781416 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:32.781460 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.781462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.781471 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.781586 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.782241 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.782249 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.782249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.782255 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.782295 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.782326 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.782421 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.782782 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.782799 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260808 07:10:32.783505 5217 tablet_service.cc:1467] Processing CreateTablet for tablet 3108c419e2d34108a70335d4b07521a2 (DEFAULT_TABLE table=test-table [id=b9c4db32ec5d4e3f802f3f5159d08bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:32.783535 5583 tablet_service.cc:1467] Processing CreateTablet for tablet 3108c419e2d34108a70335d4b07521a2 (DEFAULT_TABLE table=test-table [id=b9c4db32ec5d4e3f802f3f5159d08bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:32.783860 5583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3108c419e2d34108a70335d4b07521a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:32.783862 5217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3108c419e2d34108a70335d4b07521a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:32.786242 5682 tablet_bootstrap.cc:492] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: Bootstrap starting.
I20260808 07:10:32.786638 5683 tablet_bootstrap.cc:492] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7: Bootstrap starting.
I20260808 07:10:32.786655 5674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.781094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:40427 (local address 127.0.0.1:39342)
0808 07:10:32.781637 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.781646 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.781678 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.782629 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.782630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.782632 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:32.783063 (+ 431us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:32.783064 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:32.783071 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:32.783087 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:32.786508 (+ 3421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:32.786509 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:32.786510 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.786564 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.786580 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":69,"threads_started":1}
I20260808 07:10:32.786940 5682 tablet_bootstrap.cc:654] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:32.786957 5681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.781506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39342 (local address 127.30.54.66:40427)
0808 07:10:32.781709 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.781713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.781722 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:10:32.781749 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.781752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.781761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.782350 (+ 589us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.783172 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.786328 (+ 3156us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.786328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.786335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.786370 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.786396 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.786474 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.786842 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.786859 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
I20260808 07:10:32.787281 5682 log.cc:824] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:32.787384 5683 tablet_bootstrap.cc:654] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:32.787686 5683 log.cc:824] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:32.787587 5403 tablet_service.cc:1467] Processing CreateTablet for tablet 3108c419e2d34108a70335d4b07521a2 (DEFAULT_TABLE table=test-table [id=b9c4db32ec5d4e3f802f3f5159d08bfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:32.787921 5403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3108c419e2d34108a70335d4b07521a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:32.787974 5682 tablet_bootstrap.cc:492] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: No bootstrap required, opened a new log
I20260808 07:10:32.788039 5682 ts_tablet_manager.cc:1397] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:10:32.788479 5683 tablet_bootstrap.cc:492] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7: No bootstrap required, opened a new log
I20260808 07:10:32.788539 5683 ts_tablet_manager.cc:1397] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:32.789757 5682 raft_consensus.cc:348] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } }
I20260808 07:10:32.789866 5682 raft_consensus.cc:374] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:32.789887 5682 raft_consensus.cc:729] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfdca9f17db2467482f8095bf8c38594, State: Initialized, Role: FOLLOWER
I20260808 07:10:32.790007 5682 consensus_queue.cc:260] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } }
I20260808 07:10:32.790244 5663 heartbeater.cc:503] Master 127.30.54.126:39879 was elected leader, sending a full tablet report...
I20260808 07:10:32.790479 5682 ts_tablet_manager.cc:1428] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 07:10:32.791543 5690 tablet_bootstrap.cc:492] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: Bootstrap starting.
I20260808 07:10:32.791491 5683 raft_consensus.cc:348] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } }
I20260808 07:10:32.791635 5683 raft_consensus.cc:374] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:32.791692 5683 raft_consensus.cc:729] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a938a8fe3a824ef082fdeead750b61b7, State: Initialized, Role: FOLLOWER
I20260808 07:10:32.791814 5683 consensus_queue.cc:260] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } }
I20260808 07:10:32.792086 5317 heartbeater.cc:503] Master 127.30.54.126:39879 was elected leader, sending a full tablet report...
I20260808 07:10:32.792236 5690 tablet_bootstrap.cc:654] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:32.792276 5683 ts_tablet_manager.cc:1428] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:10:32.792603 5690 log.cc:824] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:32.793327 5690 tablet_bootstrap.cc:492] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: No bootstrap required, opened a new log
I20260808 07:10:32.793380 5690 ts_tablet_manager.cc:1397] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:32.795291 5690 raft_consensus.cc:348] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } }
I20260808 07:10:32.795404 5690 raft_consensus.cc:374] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:32.795428 5690 raft_consensus.cc:729] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a500a092fd340df8c74fa5ec3b719ea, State: Initialized, Role: FOLLOWER
I20260808 07:10:32.795557 5690 consensus_queue.cc:260] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } }
I20260808 07:10:32.795756 5690 ts_tablet_manager.cc:1428] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:10:32.795914 5491 heartbeater.cc:503] Master 127.30.54.126:39879 was elected leader, sending a full tablet report...
W20260808 07:10:32.897729 5493 tablet.cc:2369] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:32.913357 5691 raft_consensus.cc:482] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:32.913580 5691 raft_consensus.cc:504] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } }
I20260808 07:10:32.913993 5691 leader_election.cc:288] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a500a092fd340df8c74fa5ec3b719ea (127.30.54.66:40427), bfdca9f17db2467482f8095bf8c38594 (127.30.54.67:38059)
I20260808 07:10:32.915426 5319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.914166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:40427 (local address 127.30.54.65:40873)
0808 07:10:32.914236 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.914249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.914298 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.914470 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.914471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.914473 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:32.915000 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:32.915001 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:32.915010 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:32.915026 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:32.915267 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:32.915268 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:32.915269 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.915329 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.915344 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 07:10:32.915470 5681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.914257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:40873 (local address 127.30.54.66:40427)
0808 07:10:32.914303 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.914305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.914343 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:10:32.914373 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.914374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.914379 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.914460 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.915123 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.915125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.915125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.915130 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.915152 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.915157 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.915222 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.915350 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.915363 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:10:32.915565 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.914190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:38059 (local address 127.30.54.65:56403)
0808 07:10:32.914365 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.914382 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:32.914452 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:32.914631 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:32.914632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:32.914650 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:32.915176 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:32.915177 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:32.915184 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:32.915198 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:32.915406 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:32.915406 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:32.915407 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:10:32.915460 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:10:32.915483 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":98,"threads_started":1}
I20260808 07:10:32.915557 5677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.914288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:56403 (local address 127.30.54.67:38059)
0808 07:10:32.914336 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.914338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.914442 (+ 104us) server_negotiation.cc:407] Connection header received
0808 07:10:32.914544 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.914545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.914551 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.914625 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.915287 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.915289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.915289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.915295 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.915334 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.915340 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.915398 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.915460 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.915474 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:10:32.915853 5431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3108c419e2d34108a70335d4b07521a2" candidate_uuid: "a938a8fe3a824ef082fdeead750b61b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a500a092fd340df8c74fa5ec3b719ea" is_pre_election: true
I20260808 07:10:32.916024 5431 raft_consensus.cc:2454] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a938a8fe3a824ef082fdeead750b61b7 in term 0.
I20260808 07:10:32.916097 5610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3108c419e2d34108a70335d4b07521a2" candidate_uuid: "a938a8fe3a824ef082fdeead750b61b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfdca9f17db2467482f8095bf8c38594" is_pre_election: true
I20260808 07:10:32.916246 5610 raft_consensus.cc:2454] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a938a8fe3a824ef082fdeead750b61b7 in term 0.
I20260808 07:10:32.916272 5156 leader_election.cc:302] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a500a092fd340df8c74fa5ec3b719ea, a938a8fe3a824ef082fdeead750b61b7; no voters:
I20260808 07:10:32.916404 5691 raft_consensus.cc:2781] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:32.916481 5691 raft_consensus.cc:482] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:32.916507 5691 raft_consensus.cc:3037] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:32.917248 5691 raft_consensus.cc:504] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } }
I20260808 07:10:32.917383 5691 leader_election.cc:288] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4a500a092fd340df8c74fa5ec3b719ea (127.30.54.66:40427), bfdca9f17db2467482f8095bf8c38594 (127.30.54.67:38059)
I20260808 07:10:32.917572 5431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3108c419e2d34108a70335d4b07521a2" candidate_uuid: "a938a8fe3a824ef082fdeead750b61b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a500a092fd340df8c74fa5ec3b719ea"
I20260808 07:10:32.917593 5610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3108c419e2d34108a70335d4b07521a2" candidate_uuid: "a938a8fe3a824ef082fdeead750b61b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfdca9f17db2467482f8095bf8c38594"
I20260808 07:10:32.917676 5431 raft_consensus.cc:3037] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:32.917672 5610 raft_consensus.cc:3037] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:32.918463 5431 raft_consensus.cc:2454] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a938a8fe3a824ef082fdeead750b61b7 in term 1.
I20260808 07:10:32.918618 5610 raft_consensus.cc:2454] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a938a8fe3a824ef082fdeead750b61b7 in term 1.
I20260808 07:10:32.918649 5156 leader_election.cc:302] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a500a092fd340df8c74fa5ec3b719ea, a938a8fe3a824ef082fdeead750b61b7; no voters:
I20260808 07:10:32.918752 5691 raft_consensus.cc:2781] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:32.918972 5691 raft_consensus.cc:686] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 1 LEADER]: Becoming Leader. State: Replica: a938a8fe3a824ef082fdeead750b61b7, State: Running, Role: LEADER
I20260808 07:10:32.919078 5691 consensus_queue.cc:237] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } }
I20260808 07:10:32.919919 5044 catalog_manager.cc:5495] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a938a8fe3a824ef082fdeead750b61b7 (127.30.54.65). New cstate: current_term: 1 leader_uuid: "a938a8fe3a824ef082fdeead750b61b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a938a8fe3a824ef082fdeead750b61b7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:32.929109 5680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.928105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.30.54.65:43733)
0808 07:10:32.928150 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.928152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.928162 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:32.928219 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.928220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.928232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.928304 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.928905 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.928906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.928907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:32.928911 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.928925 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:32.928929 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.928977 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.929028 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.929040 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:10:32.930753 5610 raft_consensus.cc:1261] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [term 1 FOLLOWER]: Refusing update from remote peer a938a8fe3a824ef082fdeead750b61b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:32.930758 5431 raft_consensus.cc:1261] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [term 1 FOLLOWER]: Refusing update from remote peer a938a8fe3a824ef082fdeead750b61b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:32.931161 5699 consensus_queue.cc:1035] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfdca9f17db2467482f8095bf8c38594" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:32.931277 5691 consensus_queue.cc:1035] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a500a092fd340df8c74fa5ec3b719ea" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:32.933751 5703 mvcc.cc:205] Tried to move back new op lower bound from 7316164742881755136 to 7316164742837100544. Current Snapshot: MvccSnapshot[applied={T|T < 7316164742881755136}]
I20260808 07:10:32.935765 5041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:32.935844 5041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:32.936604 5041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 445b67b1098f42d1b3e2a8a3195a6ea8: Sending DeleteTablet for 3 replicas of tablet 3108c419e2d34108a70335d4b07521a2
I20260808 07:10:32.936944 5217 tablet_service.cc:1514] Processing DeleteTablet for tablet 3108c419e2d34108a70335d4b07521a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:32 UTC) from {username='slave'} at 127.0.0.1:58966
I20260808 07:10:32.936939 5403 tablet_service.cc:1514] Processing DeleteTablet for tablet 3108c419e2d34108a70335d4b07521a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:32 UTC) from {username='slave'} at 127.0.0.1:39342
I20260808 07:10:32.936980 5583 tablet_service.cc:1514] Processing DeleteTablet for tablet 3108c419e2d34108a70335d4b07521a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:32 UTC) from {username='slave'} at 127.0.0.1:43076
I20260808 07:10:32.937170 5718 tablet_replica.cc:330] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7: stopping tablet replica
I20260808 07:10:32.937268 5718 raft_consensus.cc:2229] T 3108c419e2d34108a70335d4b07521a2 P a938a8fe3a824ef082fdeead750b61b7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:32.937266 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 5129
I20260808 07:10:32.951609 5717 tablet_replica.cc:330] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: stopping tablet replica
I20260808 07:10:32.951740 5717 raft_consensus.cc:2229] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:32.951845 5717 raft_consensus.cc:2258] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:32.951999 5714 tablet_replica.cc:330] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: stopping tablet replica
I20260808 07:10:32.952087 5714 raft_consensus.cc:2229] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:32.952170 5714 raft_consensus.cc:2258] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:32.954340 5717 ts_tablet_manager.cc:1905] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:32.955116 5714 ts_tablet_manager.cc:1905] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:32.956338 5717 ts_tablet_manager.cc:1918] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:32.956408 5717 log.cc:1196] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/3108c419e2d34108a70335d4b07521a2
I20260808 07:10:32.956624 5717 ts_tablet_manager.cc:1939] T 3108c419e2d34108a70335d4b07521a2 P bfdca9f17db2467482f8095bf8c38594: Deleting consensus metadata
I20260808 07:10:32.957073 5019 catalog_manager.cc:4841] TS bfdca9f17db2467482f8095bf8c38594 (127.30.54.67:38059): tablet 3108c419e2d34108a70335d4b07521a2 (table test-table [id=b9c4db32ec5d4e3f802f3f5159d08bfe]) successfully deleted
I20260808 07:10:32.958036 5714 ts_tablet_manager.cc:1918] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:32.958118 5714 log.cc:1196] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/3108c419e2d34108a70335d4b07521a2
I20260808 07:10:32.958432 5714 ts_tablet_manager.cc:1939] T 3108c419e2d34108a70335d4b07521a2 P 4a500a092fd340df8c74fa5ec3b719ea: Deleting consensus metadata
I20260808 07:10:32.958946 5020 catalog_manager.cc:4841] TS 4a500a092fd340df8c74fa5ec3b719ea (127.30.54.66:40427): tablet 3108c419e2d34108a70335d4b07521a2 (table test-table [id=b9c4db32ec5d4e3f802f3f5159d08bfe]) successfully deleted
W20260808 07:10:32.972851 5020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:43733 (error 108)
I20260808 07:10:32.973112 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 5320
I20260808 07:10:32.975867 5674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.973250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:43733 (local address 127.0.0.1:58986)
0808 07:10:32.975750 (+ 2500us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:32.975818 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2469}
W20260808 07:10:32.977772 5020 catalog_manager.cc:4584] TS a938a8fe3a824ef082fdeead750b61b7 (127.30.54.65:43733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3108c419e2d34108a70335d4b07521a2: Network error: Client connection negotiation failed: client connection to 127.30.54.65:43733: connect: Connection refused (error 111)
I20260808 07:10:32.984163 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 5495
I20260808 07:10:32.992391 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 4996
2026-08-08T07:10:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (744 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 07:10:33.016283 30937 mini_oidc.cc:152] Starting JWKS server
I20260808 07:10:33.017510 30937 webserver.cc:460] Webserver started at https://127.0.0.1:36519/ using document root <none> and password file <none>
I20260808 07:10:33.017629 30937 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:10:33.017833 30937 webserver.cc:460] Webserver started at http://127.0.0.1:45835/ using document root <none> and password file <none>
2026-08-08T07:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:33Z Disabled control of system clock
I20260808 07:10:33.026119 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43411
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:44353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:43411
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36519/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/testchainca.cert with env {}
W20260808 07:10:33.105453 5728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:33.105648 5728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:33.105671 5728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:33.106809 5728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:33.106871 5728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:33.106884 5728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:33.106896 5728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:33.106909 5728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:10:33.106963 5728 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:10:33.106980 5728 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36519/jwks.json
I20260808 07:10:33.108474 5728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:44353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:43411
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36519/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:33.108714 5728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:33.108908 5728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:33.113584 5734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:33.113584 5738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:33.113642 5735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:33.113840 5728 server_base.cc:1034] running on GCE node
I20260808 07:10:33.114023 5728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:33.114284 5728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:33.115433 5728 hybrid_clock.cc:648] HybridClock initialized: now 1786173033115417 us; error 31 us; skew 500 ppm
I20260808 07:10:33.116705 5728 webserver.cc:460] Webserver started at http://127.30.54.126:37709/ using document root <none> and password file <none>
I20260808 07:10:33.116946 5728 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:33.116994 5728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:33.117148 5728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:33.118206 5728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "07448bc5d82e4444aac283b04902be30"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "350cf454649bd9a763aae340ba1547bf"
server_key_iv: "64291189c0aff9dcd4e82402252c13ea"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.118633 5728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "07448bc5d82e4444aac283b04902be30"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "350cf454649bd9a763aae340ba1547bf"
server_key_iv: "64291189c0aff9dcd4e82402252c13ea"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.120533 5728 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 07:10:33.121496 5747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:33.121667 5728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:33.121760 5728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "07448bc5d82e4444aac283b04902be30"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "350cf454649bd9a763aae340ba1547bf"
server_key_iv: "64291189c0aff9dcd4e82402252c13ea"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.121835 5728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:33.138024 5728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:33.138255 5728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:33.138352 5728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:33.142306 5728 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:43411
I20260808 07:10:33.142359 5814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:43411 every 8 connection(s)
I20260808 07:10:33.142684 5728 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:33.143652 5815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:33.145814 5815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30: Bootstrap starting.
I20260808 07:10:33.146394 5815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:33.146698 5815 log.cc:824] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:33.147369 5815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30: No bootstrap required, opened a new log
I20260808 07:10:33.148891 5815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07448bc5d82e4444aac283b04902be30" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43411 } }
I20260808 07:10:33.149045 5815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:33.149096 5815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07448bc5d82e4444aac283b04902be30, State: Initialized, Role: FOLLOWER
I20260808 07:10:33.149256 5815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07448bc5d82e4444aac283b04902be30" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43411 } }
I20260808 07:10:33.149335 5815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:33.149380 5815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:33.149430 5815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:33.149917 5815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07448bc5d82e4444aac283b04902be30" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43411 } }
I20260808 07:10:33.150051 5815 leader_election.cc:302] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07448bc5d82e4444aac283b04902be30; no voters:
I20260808 07:10:33.150252 5815 leader_election.cc:288] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:33.150249 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 5728
I20260808 07:10:33.150288 5818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:33.150398 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:33.150409 5818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [term 1 LEADER]: Becoming Leader. State: Replica: 07448bc5d82e4444aac283b04902be30, State: Running, Role: LEADER
I20260808 07:10:33.150492 5818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07448bc5d82e4444aac283b04902be30" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43411 } }
I20260808 07:10:33.150520 5815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:33.150626 30937 external_mini_cluster.cc:1428] Setting key 1f26de7e4eb1f38d4980c96a903f6d95
I20260808 07:10:33.151046 5819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07448bc5d82e4444aac283b04902be30. Latest consensus state: current_term: 1 leader_uuid: "07448bc5d82e4444aac283b04902be30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07448bc5d82e4444aac283b04902be30" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43411 } } }
I20260808 07:10:33.151139 5819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:33.151124 5823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07448bc5d82e4444aac283b04902be30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07448bc5d82e4444aac283b04902be30" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43411 } } }
I20260808 07:10:33.151192 5823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:33.151772 5832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:33.152386 5832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:33.153906 5832 catalog_manager.cc:1347] Generated new cluster ID: 01c746cce3654fcfad4adbfefda50d11
I20260808 07:10:33.153960 5832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:33.157399 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:33.151751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51508 (local address 127.30.54.126:43411)
0808 07:10:33.151955 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:33.151973 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:33.152215 (+ 242us) server_negotiation.cc:407] Connection header received
0808 07:10:33.152333 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:33.152340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:33.152407 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:33.152546 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:33.152734 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.154127 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.155289 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.155921 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.156058 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.156716 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:33.156720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:33.156721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:33.156918 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:33.156961 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:33.156988 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:33.157060 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:33.157130 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:10:33.157204 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260808 07:10:33.158290 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:33.163120 5832 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:33.163795 5832 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:33.169308 5832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07448bc5d82e4444aac283b04902be30: Generated new TSK 0
I20260808 07:10:33.169457 5832 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:10:33.192600 5865 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 07:10:33.192736 5865 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 07:10:33.192993 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:33.177547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.30.54.126:43411)
0808 07:10:33.177600 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:33.177602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:33.177734 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:10:33.177794 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:33.177795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:33.177806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:33.177887 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:10:33.178006 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.178753 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.179513 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.179813 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.179882 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.180057 (+ 175us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:10:33.192740 (+ 12683us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:10:33.192919 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51510: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 07:10:33.193046 5836 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51510: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 07:10:33.193300 5863 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:10:33.177469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43411 (local address 127.0.0.1:51510)
0808 07:10:33.177667 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:33.177677 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:33.177730 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:33.177887 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:33.177887 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:33.177889 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:10:33.177946 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:33.177948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.178867 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.178868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:33.179434 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:33.179436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.179887 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.179888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:33.179935 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.179953 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:10:33.192904 (+ 12951us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:10:33.193013 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.54.126:43411: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.54.126:43411: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 07:10:33.193737 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 5728
2026-08-08T07:10:33Z chronyd exiting
I20260808 07:10:33.240124 30937 test_util.cc:175] -----------------------------------------------
I20260808 07:10:33.240204 30937 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786173009519897-30937-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (224 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 07:10:33.240908 30937 mini_oidc.cc:152] Starting JWKS server
I20260808 07:10:33.241549 30937 webserver.cc:460] Webserver started at https://127.0.0.1:43913/ using document root <none> and password file <none>
I20260808 07:10:33.241597 30937 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:10:33.241740 30937 webserver.cc:460] Webserver started at http://127.0.0.1:46773/ using document root <none> and password file <none>
2026-08-08T07:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:33Z Disabled control of system clock
I20260808 07:10:33.250496 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43445
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:43445
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43913/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/testchainca.cert with env {}
W20260808 07:10:33.331581 5872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:33.331773 5872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:33.331794 5872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:33.332873 5872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:33.332908 5872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:33.332921 5872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:33.332934 5872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:33.332947 5872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:10:33.332958 5872 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:10:33.332970 5872 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43913/jwks.json
I20260808 07:10:33.334363 5872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:43445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43913/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:33.334545 5872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:33.334692 5872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:33.339545 5879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:33.339506 5880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:33.339495 5882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:33.339787 5872 server_base.cc:1034] running on GCE node
I20260808 07:10:33.340034 5872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:33.340260 5872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:33.341394 5872 hybrid_clock.cc:648] HybridClock initialized: now 1786173033341379 us; error 28 us; skew 500 ppm
I20260808 07:10:33.342676 5872 webserver.cc:460] Webserver started at http://127.30.54.126:33415/ using document root <none> and password file <none>
I20260808 07:10:33.342934 5872 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:33.342988 5872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:33.343122 5872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:33.344134 5872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "563098d56fa940fb9419eb52fef1cf35"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "2712fc11ed4ae970a6528e1b2e22f5c4"
server_key_iv: "43fd1d6103720b698f500c76a541b698"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.344513 5872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "563098d56fa940fb9419eb52fef1cf35"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "2712fc11ed4ae970a6528e1b2e22f5c4"
server_key_iv: "43fd1d6103720b698f500c76a541b698"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.345790 5872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:33.346596 5891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:33.346808 5872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:33.346944 5872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "563098d56fa940fb9419eb52fef1cf35"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "2712fc11ed4ae970a6528e1b2e22f5c4"
server_key_iv: "43fd1d6103720b698f500c76a541b698"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.347025 5872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:33.357049 5872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:33.357272 5872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:33.357379 5872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:33.361167 5872 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:43445
I20260808 07:10:33.361215 5974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:43445 every 8 connection(s)
I20260808 07:10:33.361531 5872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:33.362183 5975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:33.364719 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 5872
I20260808 07:10:33.364836 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:33.365021 5975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35: Bootstrap starting.
I20260808 07:10:33.365046 30937 external_mini_cluster.cc:1428] Setting key 0d38d63bc760c35a8c78a4310408dfee
I20260808 07:10:33.365736 5975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:33.366042 5975 log.cc:824] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:33.366783 5975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35: No bootstrap required, opened a new log
I20260808 07:10:33.368872 5975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563098d56fa940fb9419eb52fef1cf35" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43445 } }
I20260808 07:10:33.369026 5975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:33.369051 5975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563098d56fa940fb9419eb52fef1cf35, State: Initialized, Role: FOLLOWER
I20260808 07:10:33.369189 5975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563098d56fa940fb9419eb52fef1cf35" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43445 } }
I20260808 07:10:33.369266 5975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:33.369287 5975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:33.369355 5975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:33.370074 5975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563098d56fa940fb9419eb52fef1cf35" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43445 } }
I20260808 07:10:33.370198 5975 leader_election.cc:302] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 563098d56fa940fb9419eb52fef1cf35; no voters:
I20260808 07:10:33.370360 5975 leader_election.cc:288] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:33.370383 5983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:33.370496 5983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [term 1 LEADER]: Becoming Leader. State: Replica: 563098d56fa940fb9419eb52fef1cf35, State: Running, Role: LEADER
I20260808 07:10:33.370577 5983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563098d56fa940fb9419eb52fef1cf35" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43445 } }
I20260808 07:10:33.370630 5975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:33.370935 5984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "563098d56fa940fb9419eb52fef1cf35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563098d56fa940fb9419eb52fef1cf35" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43445 } } }
I20260808 07:10:33.371031 5984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:33.371196 5985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 563098d56fa940fb9419eb52fef1cf35. Latest consensus state: current_term: 1 leader_uuid: "563098d56fa940fb9419eb52fef1cf35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563098d56fa940fb9419eb52fef1cf35" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43445 } } }
I20260808 07:10:33.371261 5985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:33.371287 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:33.365546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.30.54.126:43445)
0808 07:10:33.365757 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:33.365766 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:33.365814 (+ 48us) server_negotiation.cc:407] Connection header received
0808 07:10:33.366335 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:33.366341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:33.366399 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:33.366517 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:33.366738 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.367697 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.368898 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.369485 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.369576 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.370323 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:33.370326 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:33.370327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:33.370555 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:33.370602 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:33.370619 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:33.370941 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:33.370975 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 07:10:33.371054 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260808 07:10:33.371443 5989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:33.371932 5989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:33.372174 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:33.373730 5989 catalog_manager.cc:1347] Generated new cluster ID: 8e5fda21585840c98e579c7a2259c7c9
I20260808 07:10:33.373792 5989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:33.383217 5989 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:33.383735 5989 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:33.386945 5989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 563098d56fa940fb9419eb52fef1cf35: Generated new TSK 0
I20260808 07:10:33.387122 5989 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:10:33.407987 6015 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 07:10:33.408082 6015 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 07:10:33.408378 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:33.394057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.30.54.126:43445)
0808 07:10:33.394118 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:33.394120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:33.394174 (+ 54us) server_negotiation.cc:407] Connection header received
0808 07:10:33.394238 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:33.394239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:33.394251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:33.394342 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:10:33.394428 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.395283 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.396047 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.396368 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.396440 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.396601 (+ 161us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:10:33.408169 (+ 11568us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43913/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:10:33.408308 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41986: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43913/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 07:10:33.408435 5981 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41986: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43913/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 07:10:33.408469 6013 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:10:33.393950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43445 (local address 127.0.0.1:41986)
0808 07:10:33.394116 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:33.394134 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:33.394184 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:33.394341 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:33.394342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:33.394343 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:10:33.394389 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:33.394391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.395368 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.395369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:33.395958 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:33.395960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.396454 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.396455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:33.396497 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.396502 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:10:33.408277 (+ 11775us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43913/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:10:33.408366 (+ 89us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.54.126:43445: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43913/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260808 07:10:33.408776 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 5872
2026-08-08T07:10:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (210 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 07:10:33.451020 30937 mini_oidc.cc:152] Starting JWKS server
I20260808 07:10:33.451687 30937 webserver.cc:460] Webserver started at https://127.0.0.1:46717/ using document root <none> and password file <none>
I20260808 07:10:33.451763 30937 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:10:33.451913 30937 webserver.cc:460] Webserver started at http://127.0.0.1:35913/ using document root <none> and password file <none>
2026-08-08T07:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:33Z Disabled control of system clock
I20260808 07:10:33.460680 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:35557
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:34077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:35557
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46717/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:10:33.542644 6025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:33.542877 6025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:33.542948 6025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:33.544121 6025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:33.544162 6025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:33.544175 6025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:33.544188 6025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:33.544224 6025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:10:33.544250 6025 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:10:33.544271 6025 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46717/jwks.json
I20260808 07:10:33.545676 6025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:35557
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:35557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46717/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:33.545919 6025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:33.546204 6025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:33.551028 6032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:33.551047 6033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:33.551003 6035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:33.551170 6025 server_base.cc:1034] running on GCE node
I20260808 07:10:33.551458 6025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:33.551757 6025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:33.552938 6025 hybrid_clock.cc:648] HybridClock initialized: now 1786173033552924 us; error 27 us; skew 500 ppm
I20260808 07:10:33.554127 6025 webserver.cc:460] Webserver started at http://127.30.54.126:37591/ using document root <none> and password file <none>
I20260808 07:10:33.554382 6025 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:33.554451 6025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:33.554642 6025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:33.555732 6025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "492138448eba4db3b6d858d30f2327f4"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "e6d9273c5eea4dd0c3e0dfe3dfa182c1"
server_key_iv: "3e6e1c3363f398d815fa4cfcf18edc4b"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.556018 6025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "492138448eba4db3b6d858d30f2327f4"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "e6d9273c5eea4dd0c3e0dfe3dfa182c1"
server_key_iv: "3e6e1c3363f398d815fa4cfcf18edc4b"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.557226 6025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:33.557874 6043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:33.558065 6025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:33.558132 6025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "492138448eba4db3b6d858d30f2327f4"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "e6d9273c5eea4dd0c3e0dfe3dfa182c1"
server_key_iv: "3e6e1c3363f398d815fa4cfcf18edc4b"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.558182 6025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:33.577003 6025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:33.577234 6025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:33.577342 6025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:33.581499 6025 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:35557
I20260808 07:10:33.581550 6114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:35557 every 8 connection(s)
I20260808 07:10:33.581853 6025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:33.582554 6115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:33.585074 6115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4: Bootstrap starting.
I20260808 07:10:33.585096 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 6025
I20260808 07:10:33.585203 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:33.585386 30937 external_mini_cluster.cc:1428] Setting key ccf30d1674c067fae9caf5c9f58ba8eb
I20260808 07:10:33.585741 6115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:33.586081 6115 log.cc:824] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:33.586932 6115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4: No bootstrap required, opened a new log
I20260808 07:10:33.588981 6115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492138448eba4db3b6d858d30f2327f4" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35557 } }
I20260808 07:10:33.589147 6115 raft_consensus.cc:374] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:33.589197 6115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492138448eba4db3b6d858d30f2327f4, State: Initialized, Role: FOLLOWER
I20260808 07:10:33.589322 6115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492138448eba4db3b6d858d30f2327f4" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35557 } }
I20260808 07:10:33.589416 6115 raft_consensus.cc:388] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:33.589464 6115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:33.589519 6115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:33.590168 6115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492138448eba4db3b6d858d30f2327f4" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35557 } }
I20260808 07:10:33.590292 6115 leader_election.cc:302] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 492138448eba4db3b6d858d30f2327f4; no voters:
I20260808 07:10:33.590462 6115 leader_election.cc:288] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:33.590531 6122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:33.590683 6122 raft_consensus.cc:686] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [term 1 LEADER]: Becoming Leader. State: Replica: 492138448eba4db3b6d858d30f2327f4, State: Running, Role: LEADER
I20260808 07:10:33.590724 6115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:33.590775 6122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492138448eba4db3b6d858d30f2327f4" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35557 } }
I20260808 07:10:33.591104 6122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 492138448eba4db3b6d858d30f2327f4. Latest consensus state: current_term: 1 leader_uuid: "492138448eba4db3b6d858d30f2327f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492138448eba4db3b6d858d30f2327f4" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35557 } } }
I20260808 07:10:33.591198 6122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:33.591122 6125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "492138448eba4db3b6d858d30f2327f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492138448eba4db3b6d858d30f2327f4" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35557 } } }
I20260808 07:10:33.591325 6125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:33.591413 6135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:33.591605 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:33.585879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.30.54.126:35557)
0808 07:10:33.586082 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:33.586089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:33.586113 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:10:33.586705 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:33.586713 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:33.586771 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:33.586933 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:33.587179 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.588036 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.589184 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.589769 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.589865 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.590592 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:33.590596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:33.590596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:33.590790 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:33.590863 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:33.590878 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:33.590960 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:33.591230 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 07:10:33.591320 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":104,"threads_started":1}
I20260808 07:10:33.591913 6135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:33.592525 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:33.593191 6152 mvcc.cc:205] Tried to move back new op lower bound from 7316164745595936768 to 7316164745588338688. Current Snapshot: MvccSnapshot[applied={T|T < 7316164745595936768}]
I20260808 07:10:33.593544 6135 catalog_manager.cc:1347] Generated new cluster ID: 999d782b8c104f3dab8a20bf1f618cc9
I20260808 07:10:33.593592 6135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:33.620208 6135 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:33.620786 6135 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:33.629464 6135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 492138448eba4db3b6d858d30f2327f4: Generated new TSK 0
I20260808 07:10:33.629674 6135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:33.645117 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:33.641349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.30.54.126:35557)
0808 07:10:33.641397 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:33.641399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:33.641519 (+ 120us) server_negotiation.cc:407] Connection header received
0808 07:10:33.641563 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:33.641564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:33.641577 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:33.641660 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:10:33.641784 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.642578 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.643417 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.643740 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.643811 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.643960 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:10:33.644896 (+ 936us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46717/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:10:33.645033 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34528: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46717/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 07:10:33.645179 6119 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34528: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46717/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 07:10:33.647050 6162 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:10:33.641274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:35557 (local address 127.0.0.1:34528)
0808 07:10:33.641438 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:33.641455 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:33.641513 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:33.641665 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:33.641666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:33.641668 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:10:33.641724 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:33.641726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.642680 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.642681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:33.643339 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:33.643342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.643822 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.643823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:33.643869 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.643873 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:10:33.645000 (+ 1127us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46717/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:10:33.646947 (+ 1947us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.54.126:35557: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46717/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":1722,"thread_start_us":99,"threads_started":1}
I20260808 07:10:33.647473 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 6025
E20260808 07:10:33.666563 6163 webserver.cc:533] Webserver: sslize failed:
E20260808 07:10:33.666640 6163 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T07:10:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (250 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T07:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:33Z Disabled control of system clock
I20260808 07:10:33.709522 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43083
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:43083
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/testchainca.cert with env {}
W20260808 07:10:33.791394 6169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:33.791580 6169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:33.791602 6169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:33.792676 6169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:33.792711 6169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:33.792723 6169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:33.792735 6169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:33.792747 6169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:10:33.792759 6169 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:10:33.792770 6169 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 07:10:33.794261 6169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:43083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:33.794456 6169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:33.794610 6169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:33.799299 6180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:33.799391 6177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:33.799290 6176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:33.799513 6169 server_base.cc:1034] running on GCE node
I20260808 07:10:33.799800 6169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:33.800110 6169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:33.801288 6169 hybrid_clock.cc:648] HybridClock initialized: now 1786173033801274 us; error 25 us; skew 500 ppm
I20260808 07:10:33.802505 6169 webserver.cc:460] Webserver started at http://127.30.54.126:39143/ using document root <none> and password file <none>
I20260808 07:10:33.802754 6169 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:33.802843 6169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:33.802986 6169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:33.804041 6169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "369271026b9b4f6393fc96468aa7bac6"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "9a40379f8a7d1487ab569ace66bc04bb"
server_key_iv: "6bf1b2bb32fadb446db75e849074b716"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.804401 6169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "369271026b9b4f6393fc96468aa7bac6"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "9a40379f8a7d1487ab569ace66bc04bb"
server_key_iv: "6bf1b2bb32fadb446db75e849074b716"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.805666 6169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:33.806377 6189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:33.806565 6169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:33.806658 6169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "369271026b9b4f6393fc96468aa7bac6"
format_stamp: "Formatted at 2026-08-08 07:10:33 on dist-test-slave-znh6"
server_key: "9a40379f8a7d1487ab569ace66bc04bb"
server_key_iv: "6bf1b2bb32fadb446db75e849074b716"
server_key_version: "encryptionkey@0"
I20260808 07:10:33.806739 6169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:33.819064 6169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:33.819357 6169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:33.819511 6169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:33.823468 6169 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:43083
I20260808 07:10:33.823500 6262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:43083 every 8 connection(s)
I20260808 07:10:33.823856 6169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:33.824756 6264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:33.826874 6264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6: Bootstrap starting.
I20260808 07:10:33.827384 6264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:33.827622 6264 log.cc:824] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:33.828233 6264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6: No bootstrap required, opened a new log
I20260808 07:10:33.829712 6264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369271026b9b4f6393fc96468aa7bac6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43083 } }
I20260808 07:10:33.829836 6264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:33.829859 6264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369271026b9b4f6393fc96468aa7bac6, State: Initialized, Role: FOLLOWER
I20260808 07:10:33.829965 6264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369271026b9b4f6393fc96468aa7bac6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43083 } }
I20260808 07:10:33.830015 6264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:33.830035 6264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:33.830060 6264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:33.830515 6264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369271026b9b4f6393fc96468aa7bac6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43083 } }
I20260808 07:10:33.830597 6264 leader_election.cc:302] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 369271026b9b4f6393fc96468aa7bac6; no voters:
I20260808 07:10:33.830775 6264 leader_election.cc:288] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:33.830806 6267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:33.830935 6267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [term 1 LEADER]: Becoming Leader. State: Replica: 369271026b9b4f6393fc96468aa7bac6, State: Running, Role: LEADER
I20260808 07:10:33.831027 6267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369271026b9b4f6393fc96468aa7bac6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43083 } }
I20260808 07:10:33.831060 6264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:33.831382 6269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369271026b9b4f6393fc96468aa7bac6. Latest consensus state: current_term: 1 leader_uuid: "369271026b9b4f6393fc96468aa7bac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369271026b9b4f6393fc96468aa7bac6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43083 } } }
I20260808 07:10:33.831552 6269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:33.831527 6270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "369271026b9b4f6393fc96468aa7bac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369271026b9b4f6393fc96468aa7bac6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43083 } } }
I20260808 07:10:33.831817 6270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:33.831969 6278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:33.832479 6278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:33.833899 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 6169
I20260808 07:10:33.834014 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:33.834010 6278 catalog_manager.cc:1347] Generated new cluster ID: faed3a96f20e4501bcab859b50138a26
I20260808 07:10:33.834056 6278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:33.834219 30937 external_mini_cluster.cc:1428] Setting key b06a1db5a0573ead817cb0e44c962e91
I20260808 07:10:33.840059 6299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:33.834654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40208 (local address 127.30.54.126:43083)
0808 07:10:33.834932 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:33.834941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:33.834973 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:10:33.835081 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:33.835088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:33.835156 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:33.835299 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:33.835502 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.836914 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.837929 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.838551 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.838675 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.839365 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:33.839370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:33.839371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:33.839572 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:33.839616 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:33.839630 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:33.839706 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:33.839773 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:10:33.839851 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":78,"threads_started":1}
I20260808 07:10:33.841236 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:33.848376 6278 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:33.849020 6278 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:33.851612 6278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 369271026b9b4f6393fc96468aa7bac6: Generated new TSK 0
I20260808 07:10:33.851781 6278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:33.941087 6299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:33.858266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.30.54.126:43083)
0808 07:10:33.858305 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:33.858307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:33.858383 (+ 76us) server_negotiation.cc:407] Connection header received
0808 07:10:33.858467 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:33.858468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:33.858479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:33.858545 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:10:33.858665 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.859471 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.860473 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.860796 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.860871 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.861072 (+ 201us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:10:33.940799 (+ 79727us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:10:33.941003 (+ 204us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40218: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:10:33.941172 6299 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40218: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 07:10:33.941210 6305 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:10:33.858179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43083 (local address 127.0.0.1:40218)
0808 07:10:33.858325 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:33.858335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:33.858395 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:33.858552 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:33.858553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:33.858554 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:10:33.858599 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:33.858601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.859836 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.859837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:33.860393 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:33.860395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:33.860909 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:33.860910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:33.860959 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:33.860964 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:10:33.940964 (+ 80000us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:10:33.941091 (+ 127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.54.126:43083: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":101,"threads_started":1}
I20260808 07:10:33.941802 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 6169
2026-08-08T07:10:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (263 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:34 dist-test-slave-znh6 krb5kdc[6319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:34 dist-test-slave-znh6 krb5kdc[6319](info): set up 2 sockets
Aug 08 07:10:34 dist-test-slave-znh6 krb5kdc[6319](info): commencing operation
krb5kdc: starting...
W20260808 07:10:35.357558 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:35 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:10:35.482170 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41161
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:41161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:35.561970 6337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:35.562204 6337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:35.562242 6337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:35.563426 6337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:35.563493 6337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:35.563517 6337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:35.563539 6337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:35.563570 6337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:35.564998 6337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:41161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:35.565268 6337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:35.565492 6337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:35.570147 6344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:35.570178 6343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:35.570192 6347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:35.570650 6337 server_base.cc:1034] running on GCE node
I20260808 07:10:35.570947 6337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:35.571220 6337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:35.572376 6337 hybrid_clock.cc:648] HybridClock initialized: now 1786173035572355 us; error 39 us; skew 500 ppm
Aug 08 07:10:35 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173035, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:35.574766 6337 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:10:35.575208 6337 webserver.cc:460] Webserver started at http://127.30.54.126:36281/ using document root <none> and password file <none>
I20260808 07:10:35.575397 6337 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:35.575441 6337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:35.575526 6337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:35.576505 6337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "de9cea4d0d364c73b935509974b91485"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-znh6"
server_key: "3d513dc3c20160f5481c3272c8077231"
server_key_iv: "447972cfe7522efbb0edf668ac04aaa1"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.576805 6337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "de9cea4d0d364c73b935509974b91485"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-znh6"
server_key: "3d513dc3c20160f5481c3272c8077231"
server_key_iv: "447972cfe7522efbb0edf668ac04aaa1"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.578030 6337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:10:35.578676 6356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:35.578850 6337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:35.578933 6337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "de9cea4d0d364c73b935509974b91485"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-znh6"
server_key: "3d513dc3c20160f5481c3272c8077231"
server_key_iv: "447972cfe7522efbb0edf668ac04aaa1"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.578997 6337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:35.602025 6337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:35.603893 6337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:35.604010 6337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:35.607916 6337 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:41161
I20260808 07:10:35.607975 6422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:41161 every 8 connection(s)
I20260808 07:10:35.608331 6337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:35.609217 6423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:35.611456 6423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485: Bootstrap starting.
I20260808 07:10:35.612033 6423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:35.612336 6423 log.cc:824] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:35.613008 6423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485: No bootstrap required, opened a new log
I20260808 07:10:35.614557 6423 raft_consensus.cc:348] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9cea4d0d364c73b935509974b91485" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41161 } }
I20260808 07:10:35.614717 6423 raft_consensus.cc:374] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:35.614770 6423 raft_consensus.cc:729] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9cea4d0d364c73b935509974b91485, State: Initialized, Role: FOLLOWER
I20260808 07:10:35.614941 6423 consensus_queue.cc:260] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9cea4d0d364c73b935509974b91485" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41161 } }
I20260808 07:10:35.615024 6423 raft_consensus.cc:388] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:35.615063 6423 raft_consensus.cc:482] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:35.615114 6423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:35.615628 6423 raft_consensus.cc:504] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9cea4d0d364c73b935509974b91485" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41161 } }
I20260808 07:10:35.615744 6423 leader_election.cc:302] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de9cea4d0d364c73b935509974b91485; no voters:
I20260808 07:10:35.615931 6423 leader_election.cc:288] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:35.615955 6426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:35.616201 6423 sys_catalog.cc:564] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:35.616281 6426 raft_consensus.cc:686] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [term 1 LEADER]: Becoming Leader. State: Replica: de9cea4d0d364c73b935509974b91485, State: Running, Role: LEADER
I20260808 07:10:35.616361 6426 consensus_queue.cc:237] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9cea4d0d364c73b935509974b91485" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41161 } }
I20260808 07:10:35.616735 6428 sys_catalog.cc:455] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de9cea4d0d364c73b935509974b91485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9cea4d0d364c73b935509974b91485" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41161 } } }
I20260808 07:10:35.616812 6428 sys_catalog.cc:458] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:35.616844 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 6337
I20260808 07:10:35.616954 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:35.616976 6429 sys_catalog.cc:455] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de9cea4d0d364c73b935509974b91485. Latest consensus state: current_term: 1 leader_uuid: "de9cea4d0d364c73b935509974b91485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9cea4d0d364c73b935509974b91485" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41161 } } }
I20260808 07:10:35.617031 6429 sys_catalog.cc:458] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:35.617153 30937 external_mini_cluster.cc:1428] Setting key 177b17e9e82b4adf62361858e22d581b
I20260808 07:10:35.617282 6440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:35.617808 6440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:35.619706 6440 catalog_manager.cc:1347] Generated new cluster ID: 541725b3113547c5b38479497be94c10
I20260808 07:10:35.619760 6440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:35 dist-test-slave-znh6 krb5kdc[6319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:35.627314 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:35.617636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59620 (local address 127.30.54.126:41161)
0808 07:10:35.618059 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:35.618069 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:35.618101 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:10:35.618321 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:35.618326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:35.618397 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:35.618533 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:35.619357 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:35.620381 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:35.621166 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:35.621576 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:35.621693 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:35.624138 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:35.624141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:35.624142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:35.624153 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:35.625801 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:35.626172 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:35.626174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:35.626175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:35.626264 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:35.626487 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:35.626488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:35.626488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:35.626710 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:35.626902 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:35.626983 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:10:35.627123 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":67,"threads_started":1}
I20260808 07:10:35.657557 6440 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:35.658120 6440 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:35.664291 6440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de9cea4d0d364c73b935509974b91485: Generated new TSK 0
I20260808 07:10:35.664520 6440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:10:35.697111 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41161
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:35.777254 6457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:35.777451 6457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:35.777474 6457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:35.778594 6457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:35.778633 6457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:35.778684 6457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:35.780131 6457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:35.780375 6457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:35.780607 6457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:35.785601 6471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:35.785614 6468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:35.785710 6457 server_base.cc:1034] running on GCE node
W20260808 07:10:35.785614 6469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:35.786116 6457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:35.786383 6457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:35.787557 6457 hybrid_clock.cc:648] HybridClock initialized: now 1786173035787547 us; error 27 us; skew 500 ppm
Aug 08 07:10:35 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173035, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:35.789994 6457 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:10:35.790402 6457 webserver.cc:460] Webserver started at http://127.30.54.65:43595/ using document root <none> and password file <none>
I20260808 07:10:35.790637 6457 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:35.790709 6457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:35.790885 6457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:35.791862 6457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "1ff1da58051548e8b60ce308457aafb0"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-znh6"
server_key: "66a6bd70950e3373f081e4e6d262402a"
server_key_iv: "c23e55f1ca95751b19f85874749c9413"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.792222 6457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "1ff1da58051548e8b60ce308457aafb0"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-znh6"
server_key: "66a6bd70950e3373f081e4e6d262402a"
server_key_iv: "c23e55f1ca95751b19f85874749c9413"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.793500 6457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:35.794194 6488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:35.794364 6457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:35.794459 6457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "1ff1da58051548e8b60ce308457aafb0"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-znh6"
server_key: "66a6bd70950e3373f081e4e6d262402a"
server_key_iv: "c23e55f1ca95751b19f85874749c9413"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.794541 6457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:35.808085 6457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:35.809767 6457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:35.809919 6457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:35.810139 6457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:35.810427 6457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:35.810487 6457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:35.810524 6457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:35.810552 6457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:35.816159 6457 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:42719
I20260808 07:10:35.816212 6647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:42719 every 8 connection(s)
I20260808 07:10:35.816529 6457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:35.822052 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 6457
I20260808 07:10:35.822180 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:35.822403 30937 external_mini_cluster.cc:1428] Setting key 4c8c975abf241959daabceccf8486a00
Aug 08 07:10:35 dist-test-slave-znh6 krb5kdc[6319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173035, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:35.826509 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:35.817390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:43311 (local address 127.30.54.126:41161)
0808 07:10:35.817489 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:35.817491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:35.817643 (+ 152us) server_negotiation.cc:407] Connection header received
0808 07:10:35.818652 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:35.818654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:35.818666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:35.818719 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:35.819374 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:35.819894 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:35.820880 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:35.821102 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:35.821181 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:35.823482 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:35.823484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:35.823484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:35.823495 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:35.825070 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:35.825574 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:35.825575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:35.825576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:35.825628 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:35.825895 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:35.825896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:35.825896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:35.825997 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:35.826046 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:35.826370 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 07:10:35.826439 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 07:10:35.826797 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:35.817312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41161 (local address 127.30.54.65:43311)
0808 07:10:35.817511 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:35.817551 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:35.818540 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:35.818784 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:35.818788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:35.819105 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:35.819276 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:35.819281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:35.820004 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:35.820005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:35.820789 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:35.820791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:35.821188 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:35.821189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:35.821316 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:35.821741 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:35.821749 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:35.823350 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:35.825159 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:35.825166 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:35.825168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:35.825499 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:35.825691 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:35.825692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:35.825692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:35.825832 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:35.826064 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:35.826070 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:35.826235 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 07:10:35.826350 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:10:35.826530 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260808 07:10:35.827390 6648 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41161
I20260808 07:10:35.827512 6648 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:35.827702 6648 heartbeater.cc:511] Master 127.30.54.126:41161 requested a full tablet report, sending...
I20260808 07:10:35.828197 6375 ts_manager.cc:194] Registered new tserver with Master: 1ff1da58051548e8b60ce308457aafb0 (127.30.54.65:42719)
I20260808 07:10:35.828809 6375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:43311
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:10:35.850222 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41161
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:35.932821 6655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:35.933285 6655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:35.933352 6655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:35.934625 6655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:35.934665 6655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:35.934692 6655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:35.936182 6655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:35.936404 6655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:35.936635 6655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:35.943053 6662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:35.943064 6668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:35.943073 6670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:35.943521 6655 server_base.cc:1034] running on GCE node
I20260808 07:10:35.943744 6655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:35.944028 6655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:35.945204 6655 hybrid_clock.cc:648] HybridClock initialized: now 1786173035945188 us; error 27 us; skew 500 ppm
Aug 08 07:10:35 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173035, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:35.948233 6655 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:10:35.948704 6655 webserver.cc:460] Webserver started at http://127.30.54.66:41095/ using document root <none> and password file <none>
I20260808 07:10:35.948922 6655 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:35.948989 6655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:35.949142 6655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:35.950155 6655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "57c39d3464bd41d8b6407ef3de11d703"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-znh6"
server_key: "bd7c44072f209aad3b09f04ebbfcff9a"
server_key_iv: "891194302f7c7ec08aa5f288c3b1f618"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.950524 6655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "57c39d3464bd41d8b6407ef3de11d703"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-znh6"
server_key: "bd7c44072f209aad3b09f04ebbfcff9a"
server_key_iv: "891194302f7c7ec08aa5f288c3b1f618"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.951958 6655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:35.952749 6678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:35.952916 6655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:35.953022 6655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "57c39d3464bd41d8b6407ef3de11d703"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-znh6"
server_key: "bd7c44072f209aad3b09f04ebbfcff9a"
server_key_iv: "891194302f7c7ec08aa5f288c3b1f618"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.953094 6655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:35.985316 6655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:35.987740 6655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:35.987905 6655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:35.988157 6655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:35.988533 6655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:35.988586 6655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:35.988629 6655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:35.988656 6655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:35.995870 6655 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:36175
I20260808 07:10:35.995949 6836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:36175 every 8 connection(s)
I20260808 07:10:35.996249 6655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:10:35 dist-test-slave-znh6 krb5kdc[6319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173035, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:36.005371 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 6655
I20260808 07:10:36.005512 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:36.005750 30937 external_mini_cluster.cc:1428] Setting key 97566e2d050ab0871123da6491d6d5b0
I20260808 07:10:36.008045 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:35.997081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:57357 (local address 127.30.54.126:41161)
0808 07:10:35.997160 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:35.997162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:35.997406 (+ 244us) server_negotiation.cc:407] Connection header received
0808 07:10:35.998937 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:35.998938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:35.998949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:35.999002 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:35.999882 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.000375 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.002055 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.002240 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.002287 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:36.004813 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:36.004814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:36.004815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:36.004823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:36.006585 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.007062 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.007064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.007064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.007127 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.007383 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.007384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.007384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.007501 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:36.007583 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:36.007856 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 07:10:36.007956 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:10:36.008234 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:35.997054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41161 (local address 127.30.54.66:57357)
0808 07:10:35.997258 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:35.997307 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:35.998760 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:35.999079 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:35.999084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:35.999486 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:35.999782 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:35.999788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.000494 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.000495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:36.001960 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:36.001963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.002317 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.002319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:36.002490 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:36.003076 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:36.003086 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:36.004708 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:36.006669 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:36.006676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:36.006678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:36.006972 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:36.007193 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:36.007194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:36.007195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:36.007309 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:36.007573 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:36.007580 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:36.007720 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:10:36.007821 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:10:36.007981 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260808 07:10:36.009330 6841 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41161
I20260808 07:10:36.009443 6841 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:36.009655 6841 heartbeater.cc:511] Master 127.30.54.126:41161 requested a full tablet report, sending...
I20260808 07:10:36.010011 6375 ts_manager.cc:194] Registered new tserver with Master: 57c39d3464bd41d8b6407ef3de11d703 (127.30.54.66:36175)
I20260808 07:10:36.010396 6375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:57357
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:10:36.035171 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41161
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:36.114584 6851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:36.114809 6851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:36.114861 6851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:36.116030 6851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:36.116091 6851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:36.116137 6851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:36.117991 6851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:36.118249 6851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:36.118489 6851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:36.123908 6859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:36.123900 6861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:36.123921 6858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:36.124250 6851 server_base.cc:1034] running on GCE node
I20260808 07:10:36.124475 6851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:36.124682 6851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:36.125865 6851 hybrid_clock.cc:648] HybridClock initialized: now 1786173036125849 us; error 33 us; skew 500 ppm
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:36.128450 6851 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:10:36.128916 6851 webserver.cc:460] Webserver started at http://127.30.54.67:37975/ using document root <none> and password file <none>
I20260808 07:10:36.129159 6851 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:36.129225 6851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:36.129379 6851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:36.130582 6851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "9454810791fc416e80dfb1ca62a4d936"
format_stamp: "Formatted at 2026-08-08 07:10:36 on dist-test-slave-znh6"
server_key: "8a354c113d54c4c7dfd0358ed663d148"
server_key_iv: "b63e9691a2bc2ff929963d93979c5102"
server_key_version: "encryptionkey@0"
I20260808 07:10:36.131038 6851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "9454810791fc416e80dfb1ca62a4d936"
format_stamp: "Formatted at 2026-08-08 07:10:36 on dist-test-slave-znh6"
server_key: "8a354c113d54c4c7dfd0358ed663d148"
server_key_iv: "b63e9691a2bc2ff929963d93979c5102"
server_key_version: "encryptionkey@0"
I20260808 07:10:36.133087 6851 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 07:10:36.133900 6872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:36.134112 6851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:36.134229 6851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "9454810791fc416e80dfb1ca62a4d936"
format_stamp: "Formatted at 2026-08-08 07:10:36 on dist-test-slave-znh6"
server_key: "8a354c113d54c4c7dfd0358ed663d148"
server_key_iv: "b63e9691a2bc2ff929963d93979c5102"
server_key_version: "encryptionkey@0"
I20260808 07:10:36.134311 6851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:36.143991 6851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:36.145908 6851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:36.146064 6851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:36.146283 6851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:36.146584 6851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:36.146638 6851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:36.146678 6851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:36.146703 6851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:36.152926 6851 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:46449
I20260808 07:10:36.152985 7045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:46449 every 8 connection(s)
I20260808 07:10:36.153278 6851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:10:36.159835 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 6851
I20260808 07:10:36.159950 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:36.160177 30937 external_mini_cluster.cc:1428] Setting key a01f663b177eeeedf5fa1fa4fc49fb62
I20260808 07:10:36.162493 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:36.153974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:48951 (local address 127.30.54.126:41161)
0808 07:10:36.154043 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:36.154045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:36.154251 (+ 206us) server_negotiation.cc:407] Connection header received
0808 07:10:36.155195 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:36.155196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:36.155206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:36.155265 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:36.155921 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.156428 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.157458 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.157652 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.157711 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:36.159844 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:36.159846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:36.159847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:36.159855 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:36.161366 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.161737 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.161738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.161739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.161790 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.162000 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.162001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.162001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.162098 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:36.162147 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:36.162362 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 07:10:36.162427 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:10:36.162658 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:36.153960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41161 (local address 127.30.54.67:48951)
0808 07:10:36.154124 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:36.154165 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:36.155107 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:36.155314 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:36.155317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:36.155564 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:36.155837 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:36.155842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.156508 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.156509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:36.157362 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:36.157364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.157779 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.157781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:36.157908 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:36.158416 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:36.158424 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:36.159743 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:36.161452 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:36.161457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:36.161459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:36.161669 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:36.161848 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:36.161849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:36.161849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:36.161940 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:36.162151 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:36.162155 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:36.162255 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 07:10:36.162334 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:10:36.162459 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260808 07:10:36.163084 7048 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41161
I20260808 07:10:36.163165 7048 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:36.163359 7048 heartbeater.cc:511] Master 127.30.54.126:41161 requested a full tablet report, sending...
I20260808 07:10:36.163734 6375 ts_manager.cc:194] Registered new tserver with Master: 9454810791fc416e80dfb1ca62a4d936 (127.30.54.67:46449)
I20260808 07:10:36.164101 6375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:48951
I20260808 07:10:36.171895 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/test-admin.keytab.
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:36.187992 30937 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 07:10:36.189627 7059 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:10:36.188826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.30.54.65:42719)
0808 07:10:36.189008 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:36.189012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:36.189023 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:36.189050 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:36.189052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:36.189066 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:36.189141 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:36.189498 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53822: BlockingRecv error: recv got EOF from 127.0.0.1:53822 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
W20260808 07:10:36.189694 30937 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:36.190691 30937 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:36.191625 30937 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:36.199404 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:36.192139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53824 (local address 127.30.54.65:42719)
0808 07:10:36.192189 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:36.192191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:36.192229 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:10:36.192246 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:36.192246 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:36.192254 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:36.192294 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:36.192665 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.193380 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.194066 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.194366 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.194426 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:36.196551 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:36.196560 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:36.196561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:36.196574 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:36.198153 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.198539 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.198542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.198543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.198604 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.198809 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.198827 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.198827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.198971 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:36.199115 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:36.199207 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 07:10:36.199286 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 07:10:36.201278 7059 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:10:36.200688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.30.54.65:42719)
0808 07:10:36.200747 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:36.200749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:36.200818 (+ 69us) server_negotiation.cc:407] Connection header received
0808 07:10:36.200892 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:36.200893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:36.200903 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:36.200947 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:36.201233 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53838: BlockingRecv error: recv got EOF from 127.0.0.1:53838 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 07:10:36.201408 30937 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:36.202313 30937 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:36.203226 30937 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:36.209992 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:36.203743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53842 (local address 127.30.54.65:42719)
0808 07:10:36.203798 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:36.203800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:36.203852 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:10:36.203907 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:36.203908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:36.203917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:36.203971 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:36.204324 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.204816 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.205528 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.205733 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.205808 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:36.207766 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:36.207767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:36.207768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:36.207775 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:36.208910 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.209241 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.209242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.209242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.209301 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.209514 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.209515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.209515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.209616 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:36.209726 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:36.209823 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 07:10:36.209905 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 07:10:36.211444 7059 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:10:36.210950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53848 (local address 127.30.54.65:42719)
0808 07:10:36.210984 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:36.210986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:36.211055 (+ 69us) server_negotiation.cc:407] Connection header received
0808 07:10:36.211090 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:36.211091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:36.211097 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:36.211144 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:36.211407 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53848: BlockingRecv error: recv got EOF from 127.0.0.1:53848 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:10:36.211575 30937 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786173009519897-30937-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:36.212494 30937 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:36.213330 30937 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[6319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:10:36.219790 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:36.213835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53850 (local address 127.30.54.65:42719)
0808 07:10:36.213897 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:36.213899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:36.213920 (+ 21us) server_negotiation.cc:407] Connection header received
0808 07:10:36.213966 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:36.213967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:36.213973 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:36.214018 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:36.214339 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.214758 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.215438 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:36.215619 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:36.215671 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:36.217653 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:36.217655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:36.217655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:36.217660 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:36.218867 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.219182 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.219183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.219183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.219231 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:36.219391 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:36.219391 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:36.219392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:36.219472 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:36.219536 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:36.219624 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:10:36.219676 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:10:36.220394 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 6457
I20260808 07:10:36.232371 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 6655
I20260808 07:10:36.244463 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 6851
I20260808 07:10:36.253268 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 6337
2026-08-08T07:10:36Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2130 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[7078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[7078](info): set up 2 sockets
Aug 08 07:10:36 dist-test-slave-znh6 krb5kdc[7078](info): commencing operation
krb5kdc: starting...
W20260808 07:10:37.499750 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:10:37 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.126.keytab.
Entry for principal oryx/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.126.keytab.
I20260808 07:10:37.622082 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39687
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.126.keytab
--principal=oryx/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:39687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:37.702911 7094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:37.703114 7094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:37.703135 7094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:37.704223 7094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:37.704259 7094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:37.704274 7094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:37.704288 7094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:37.704299 7094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:37.705694 7094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:39687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.126.keytab
--principal=oryx/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:37.705906 7094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:37.706113 7094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:37.711189 7099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:37.711189 7104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:37.711208 7094 server_base.cc:1034] running on GCE node
W20260808 07:10:37.711185 7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:37.711649 7094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:37.711978 7094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:37.713126 7094 hybrid_clock.cc:648] HybridClock initialized: now 1786173037713114 us; error 30 us; skew 500 ppm
Aug 08 07:10:37 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:37.718643 7094 init.cc:377] Logged in from keytab as oryx/127.30.54.126@KRBTEST.COM (short username oryx)
I20260808 07:10:37.719134 7094 webserver.cc:460] Webserver started at http://127.30.54.126:44159/ using document root <none> and password file <none>
I20260808 07:10:37.719359 7094 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:37.719432 7094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:37.719583 7094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:37.720623 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "bec56ea19cb14faebd957b877001c29b"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-znh6"
server_key: "1f063039d236ba292c6ddd366fc46f29"
server_key_iv: "cdd757b19ffbe10877c8ecb8b8e09ec6"
server_key_version: "encryptionkey@0"
I20260808 07:10:37.720985 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "bec56ea19cb14faebd957b877001c29b"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-znh6"
server_key: "1f063039d236ba292c6ddd366fc46f29"
server_key_iv: "cdd757b19ffbe10877c8ecb8b8e09ec6"
server_key_version: "encryptionkey@0"
I20260808 07:10:37.722317 7094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:37.723115 7112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:37.723301 7094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:37.723402 7094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "bec56ea19cb14faebd957b877001c29b"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-znh6"
server_key: "1f063039d236ba292c6ddd366fc46f29"
server_key_iv: "cdd757b19ffbe10877c8ecb8b8e09ec6"
server_key_version: "encryptionkey@0"
I20260808 07:10:37.723489 7094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:37.732388 7094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:37.734197 7094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:37.734352 7094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:37.738400 7094 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:39687
I20260808 07:10:37.738437 7186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:39687 every 8 connection(s)
I20260808 07:10:37.738770 7094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:37.739784 7187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:37.742022 7187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b: Bootstrap starting.
I20260808 07:10:37.742602 7187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:37.742980 7187 log.cc:824] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:37.743687 7187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b: No bootstrap required, opened a new log
I20260808 07:10:37.745266 7187 raft_consensus.cc:348] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } }
I20260808 07:10:37.745404 7187 raft_consensus.cc:374] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:37.745453 7187 raft_consensus.cc:729] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec56ea19cb14faebd957b877001c29b, State: Initialized, Role: FOLLOWER
I20260808 07:10:37.745597 7187 consensus_queue.cc:260] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } }
I20260808 07:10:37.745659 7187 raft_consensus.cc:388] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:37.745682 7187 raft_consensus.cc:482] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:37.745749 7187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:37.746543 7187 raft_consensus.cc:504] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } }
I20260808 07:10:37.746660 7187 leader_election.cc:302] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bec56ea19cb14faebd957b877001c29b; no voters:
I20260808 07:10:37.746907 7187 leader_election.cc:288] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:37.746934 7190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:37.746927 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 7094
I20260808 07:10:37.747051 7190 raft_consensus.cc:686] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 1 LEADER]: Becoming Leader. State: Replica: bec56ea19cb14faebd957b877001c29b, State: Running, Role: LEADER
I20260808 07:10:37.747056 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:10:37.747136 7190 consensus_queue.cc:237] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } }
I20260808 07:10:37.747272 30937 external_mini_cluster.cc:1428] Setting key 352c1a13f81c90030647f71c45ee4503
I20260808 07:10:37.747169 7187 sys_catalog.cc:564] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:37.747641 7193 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bec56ea19cb14faebd957b877001c29b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } }
I20260808 07:10:37.747726 7193 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:37.747864 7191 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bec56ea19cb14faebd957b877001c29b. Latest consensus state: current_term: 1 leader_uuid: "bec56ea19cb14faebd957b877001c29b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } }
I20260808 07:10:37.747932 7191 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:37.748063 7198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:37.748860 7198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:37.751281 7198 catalog_manager.cc:1347] Generated new cluster ID: 8bc5edd0134b4405b0518f7483db023f
I20260808 07:10:37.751335 7198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:37 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:10:37.758206 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:37.747936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43650 (local address 127.30.54.126:39687)
0808 07:10:37.748117 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:37.748127 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:37.748406 (+ 279us) server_negotiation.cc:407] Connection header received
0808 07:10:37.748532 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:37.748541 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:37.748609 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:37.748757 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:37.749613 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.751003 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.751917 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.752355 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.752491 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:37.754859 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:37.754865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:37.754869 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:37.754882 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:37.756569 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:37.756972 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:37.756975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:37.756976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:37.757082 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:37.757310 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:37.757310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:37.757311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:37.757543 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:37.757733 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:37.757802 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:10:37.758045 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
I20260808 07:10:37.769273 7198 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:37.769834 7198 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:37.779829 7198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b: Generated new TSK 0
I20260808 07:10:37.780040 7198 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab.
Entry for principal oryx/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab.
I20260808 07:10:37.809821 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39687
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab
--principal=oryx/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:37.892549 7226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:37.892788 7226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:37.892827 7226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:37.893936 7226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:37.894006 7226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:37.894052 7226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:37.895607 7226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab
--principal=oryx/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:37.895879 7226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:37.896119 7226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:37.901454 7239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:37.901475 7226 server_base.cc:1034] running on GCE node
W20260808 07:10:37.901473 7233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:37.901443 7234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:37.901875 7226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:37.902145 7226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:37.903296 7226 hybrid_clock.cc:648] HybridClock initialized: now 1786173037903278 us; error 35 us; skew 500 ppm
Aug 08 07:10:37 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:37.905907 7226 init.cc:377] Logged in from keytab as oryx/127.30.54.65@KRBTEST.COM (short username oryx)
I20260808 07:10:37.906374 7226 webserver.cc:460] Webserver started at http://127.30.54.65:41513/ using document root <none> and password file <none>
I20260808 07:10:37.906576 7226 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:37.906617 7226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:37.906705 7226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:37.907757 7226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "762dda4727d845a5bb56adb9828d3308"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-znh6"
server_key: "d7b58158ab80505527b194027e266e13"
server_key_iv: "17cdc863390205974d485b2537d125db"
server_key_version: "encryptionkey@0"
I20260808 07:10:37.908068 7226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "762dda4727d845a5bb56adb9828d3308"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-znh6"
server_key: "d7b58158ab80505527b194027e266e13"
server_key_iv: "17cdc863390205974d485b2537d125db"
server_key_version: "encryptionkey@0"
I20260808 07:10:37.909339 7226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:37.910053 7248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:37.910246 7226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:37.910367 7226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "762dda4727d845a5bb56adb9828d3308"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-znh6"
server_key: "d7b58158ab80505527b194027e266e13"
server_key_iv: "17cdc863390205974d485b2537d125db"
server_key_version: "encryptionkey@0"
I20260808 07:10:37.910442 7226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:37.924082 7226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:37.926451 7226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:37.926610 7226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:37.926908 7226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:37.927273 7226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:37.927309 7226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:37.927331 7226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:37.927371 7226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:37.933121 7226 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:38663
I20260808 07:10:37.933202 7408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:38663 every 8 connection(s)
I20260808 07:10:37.933518 7226 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:37.935528 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 7226
I20260808 07:10:37.935670 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:37.935892 30937 external_mini_cluster.cc:1428] Setting key fd9fab7281aa7a7f0d9bbe28540c4439
Aug 08 07:10:37 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:10:37.943468 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:37.934268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:46849 (local address 127.30.54.126:39687)
0808 07:10:37.934333 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:37.934335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:37.934561 (+ 226us) server_negotiation.cc:407] Connection header received
0808 07:10:37.935125 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:37.935126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:37.935141 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:37.935202 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:37.936440 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.936960 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.937961 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.938174 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.938261 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:37.940618 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:37.940620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:37.940620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:37.940629 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:37.942024 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:37.942438 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:37.942439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:37.942440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:37.942498 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:37.942747 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:37.942748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:37.942748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:37.942916 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:37.942992 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:37.943282 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 07:10:37.943378 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:10:37.943670 7412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:37.934229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:46849)
0808 07:10:37.934436 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:37.934473 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:37.935017 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:37.935278 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:37.935281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:37.936137 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:37.936329 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:37.936334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.937057 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.937058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:37.937849 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:37.937851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.938273 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.938274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:37.938398 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:37.938967 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:37.938975 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:37.940520 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:37.942107 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:37.942114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:37.942117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:37.942336 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:37.942565 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:37.942566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:37.942566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:37.942676 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:37.942984 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:37.942991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:37.943126 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 07:10:37.943230 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 07:10:37.943399 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":122,"threads_started":1}
I20260808 07:10:37.944175 7410 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39687
I20260808 07:10:37.944262 7410 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:37.944447 7410 heartbeater.cc:511] Master 127.30.54.126:39687 requested a full tablet report, sending...
I20260808 07:10:37.945040 7137 ts_manager.cc:194] Registered new tserver with Master: 762dda4727d845a5bb56adb9828d3308 (127.30.54.65:38663)
WARNING: no policy specified for oryx/127.30.54.66@KRBTEST.COM; defaulting to no policy
I20260808 07:10:37.945886 7137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:46849
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab.
Entry for principal oryx/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab.
I20260808 07:10:37.961850 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39687
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab
--principal=oryx/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:38.047163 7415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:38.047363 7415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:38.047385 7415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:38.048609 7415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:38.048662 7415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:38.048689 7415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:38.050072 7415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab
--principal=oryx/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:38.050246 7415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:38.050420 7415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:38.055339 7422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.055485 7415 server_base.cc:1034] running on GCE node
W20260808 07:10:38.055364 7421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.055364 7428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.055806 7415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:38.056056 7415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:38.057248 7415 hybrid_clock.cc:648] HybridClock initialized: now 1786173038057228 us; error 40 us; skew 500 ppm
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:38.059767 7415 init.cc:377] Logged in from keytab as oryx/127.30.54.66@KRBTEST.COM (short username oryx)
I20260808 07:10:38.060225 7415 webserver.cc:460] Webserver started at http://127.30.54.66:39559/ using document root <none> and password file <none>
I20260808 07:10:38.060423 7415 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.060472 7415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.060562 7415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:38.061540 7415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "933fc011f83a4f16a77ddea9e4d2936d"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "230f6b57491245ee2c36b35f8d592c3b"
server_key_iv: "5349211570e3fff1e9ffc4605b283a0d"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.061838 7415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "933fc011f83a4f16a77ddea9e4d2936d"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "230f6b57491245ee2c36b35f8d592c3b"
server_key_iv: "5349211570e3fff1e9ffc4605b283a0d"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.063370 7415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:10:38.064141 7437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.064357 7415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:38.064436 7415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "933fc011f83a4f16a77ddea9e4d2936d"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "230f6b57491245ee2c36b35f8d592c3b"
server_key_iv: "5349211570e3fff1e9ffc4605b283a0d"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.064502 7415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.080669 7415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:38.082962 7415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:38.083081 7415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:38.083293 7415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:38.083622 7415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:38.083657 7415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.083700 7415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:38.083727 7415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.089560 7415 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:45907
I20260808 07:10:38.089715 7603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:45907 every 8 connection(s)
I20260808 07:10:38.089960 7415 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:10:38.096817 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 7415
I20260808 07:10:38.096935 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:38.097126 30937 external_mini_cluster.cc:1428] Setting key 0925417d63386fc4061c9975a7730611
I20260808 07:10:38.099613 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.090763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:59353 (local address 127.30.54.126:39687)
0808 07:10:38.090851 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.090853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.091030 (+ 177us) server_negotiation.cc:407] Connection header received
0808 07:10:38.091522 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.091523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.091532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.091593 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.092514 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.093021 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.094064 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.094259 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.094320 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.096439 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.096440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.096440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.096447 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.098157 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.098653 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.098654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.098654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.098707 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.099004 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.099005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.099005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.099107 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.099161 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.099483 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.099549 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:10:38.099889 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.090722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:59353)
0808 07:10:38.090901 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.090942 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.091435 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.091659 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.091662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.092247 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.092437 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.092442 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.093105 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.093106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.093968 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.093971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.094342 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.094343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.094454 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.095002 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.095018 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.096348 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.098238 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.098245 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.098247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.098575 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.098773 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.098775 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.098775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.098937 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.099173 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.099179 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.099337 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.099449 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.099620 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":113,"threads_started":1}
I20260808 07:10:38.100392 7604 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39687
I20260808 07:10:38.100497 7604 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:38.100667 7604 heartbeater.cc:511] Master 127.30.54.126:39687 requested a full tablet report, sending...
I20260808 07:10:38.101053 7137 ts_manager.cc:194] Registered new tserver with Master: 933fc011f83a4f16a77ddea9e4d2936d (127.30.54.66:45907)
I20260808 07:10:38.101437 7137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:59353
WARNING: no policy specified for oryx/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab.
Entry for principal oryx/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab.
I20260808 07:10:38.125782 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39687
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab
--principal=oryx/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:38.208804 7609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:38.209008 7609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:38.209033 7609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:38.210093 7609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:38.210127 7609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:38.210153 7609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:38.211613 7609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab
--principal=oryx/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:39687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:38.211848 7609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:38.212093 7609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:38.217255 7618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.217350 7615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.217267 7614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.217512 7609 server_base.cc:1034] running on GCE node
I20260808 07:10:38.217738 7609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:38.217984 7609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:38.219137 7609 hybrid_clock.cc:648] HybridClock initialized: now 1786173038219128 us; error 41 us; skew 500 ppm
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:38.221774 7609 init.cc:377] Logged in from keytab as oryx/127.30.54.67@KRBTEST.COM (short username oryx)
I20260808 07:10:38.222203 7609 webserver.cc:460] Webserver started at http://127.30.54.67:35913/ using document root <none> and password file <none>
I20260808 07:10:38.222424 7609 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.222493 7609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.222636 7609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:38.223791 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "cd43768cb6c141a6a81a65237b3d7420"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "a668c81a92a060e7b4c50d61ebff7431"
server_key_iv: "228df9a1cc0bc11fab362e998b31c38a"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.224174 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "cd43768cb6c141a6a81a65237b3d7420"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "a668c81a92a060e7b4c50d61ebff7431"
server_key_iv: "228df9a1cc0bc11fab362e998b31c38a"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.225495 7609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:38.226315 7630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.226495 7609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:38.226601 7609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "cd43768cb6c141a6a81a65237b3d7420"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "a668c81a92a060e7b4c50d61ebff7431"
server_key_iv: "228df9a1cc0bc11fab362e998b31c38a"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.226675 7609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.254047 7609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:38.255970 7609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:38.256130 7609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:38.256356 7609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:38.256670 7609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:38.256742 7609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.256803 7609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:38.256835 7609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.262936 7609 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:42685
I20260808 07:10:38.262981 7789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:42685 every 8 connection(s)
I20260808 07:10:38.263352 7609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:10:38.271023 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 7609
I20260808 07:10:38.271117 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:38.271306 30937 external_mini_cluster.cc:1428] Setting key 8c42e230b88a4acd9eef274bc1d55e1b
I20260808 07:10:38.273020 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.264166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:36901 (local address 127.30.54.126:39687)
0808 07:10:38.264235 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.264237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.264413 (+ 176us) server_negotiation.cc:407] Connection header received
0808 07:10:38.264915 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.264918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.264929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.265011 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.265938 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.266453 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.267770 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.267960 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.268025 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.270177 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.270179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.270179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.270185 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.271832 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.272281 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.272282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.272283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.272348 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.272571 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.272571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.272572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.272667 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.272712 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.272895 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.272957 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:10:38.273204 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.264129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:36901)
0808 07:10:38.264280 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.264320 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.264796 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.265065 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.265068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.265649 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.265838 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.265842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.266539 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.266539 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.267631 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.267634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.268043 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.268043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.268144 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.268692 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.268700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.270077 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.271889 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.271895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.271897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.272210 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.272414 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.272415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.272416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.272528 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.272704 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.272707 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.272810 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.272883 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.273002 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260808 07:10:38.273643 7790 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39687
I20260808 07:10:38.273752 7790 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:38.273926 7790 heartbeater.cc:511] Master 127.30.54.126:39687 requested a full tablet report, sending...
I20260808 07:10:38.274297 7137 ts_manager.cc:194] Registered new tserver with Master: cd43768cb6c141a6a81a65237b3d7420 (127.30.54.67:42685)
I20260808 07:10:38.274670 7137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:36901
I20260808 07:10:38.283320 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.54.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.125.keytab.
Entry for principal oryx/127.30.54.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.125.keytab.
I20260808 07:10:38.309957 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.125:39515
--webserver_interface=127.30.54.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.125.keytab
--principal=oryx/127.30.54.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:39687,127.30.54.125:39515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:38.391916 7796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:38.392146 7796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:38.392185 7796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:38.393334 7796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:38.393399 7796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:38.393424 7796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:38.393447 7796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:38.393476 7796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:38.395018 7796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:39687,127.30.54.125:39515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.125.keytab
--principal=oryx/127.30.54.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.125:39515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.54.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
W20260808 07:10:38.395188 7796 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.54.126:39687,127.30.54.125:39515'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 07:10:38.395340 7796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:38.395567 7796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:38.401932 7806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.401938 7804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.402177 7803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.402168 7796 server_base.cc:1034] running on GCE node
I20260808 07:10:38.402552 7796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:38.402987 7796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:38.404152 7796 hybrid_clock.cc:648] HybridClock initialized: now 1786173038404137 us; error 28 us; skew 500 ppm
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:38.407613 7796 init.cc:377] Logged in from keytab as oryx/127.30.54.125@KRBTEST.COM (short username oryx)
I20260808 07:10:38.408156 7796 webserver.cc:460] Webserver started at http://127.30.54.125:36101/ using document root <none> and password file <none>
I20260808 07:10:38.408453 7796 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.408530 7796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.408684 7796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:38.409958 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data/instance:
uuid: "ce4b9e938cd94ec1b42843f52bd9936e"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "5ee1dc3a8eb799b402ce9d3b106bbb26"
server_key_iv: "78d84ba9e7c0e7f356b6ce311c8753d7"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.410324 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal/instance:
uuid: "ce4b9e938cd94ec1b42843f52bd9936e"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "5ee1dc3a8eb799b402ce9d3b106bbb26"
server_key_iv: "78d84ba9e7c0e7f356b6ce311c8753d7"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.411778 7796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:38.412635 7813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.412850 7796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:38.412952 7796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal
uuid: "ce4b9e938cd94ec1b42843f52bd9936e"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "5ee1dc3a8eb799b402ce9d3b106bbb26"
server_key_iv: "78d84ba9e7c0e7f356b6ce311c8753d7"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.413040 7796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.420676 7796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:38.422385 7796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:38.422544 7796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:38.426998 7796 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.125:39515
I20260808 07:10:38.427031 7892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.125:39515 every 8 connection(s)
I20260808 07:10:38.427402 7796 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
I20260808 07:10:38.428421 7893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:38.430213 7893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:10:38.435195 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 7796
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.125@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:10:38.439811 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.430970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.30.54.126:39687)
0808 07:10:38.431054 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.431056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.431273 (+ 217us) server_negotiation.cc:407] Connection header received
0808 07:10:38.431781 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.431782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.431795 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.431847 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.432458 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.432976 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.433983 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.434194 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.434281 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.436635 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.436637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.436637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.436646 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.438358 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.438756 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.438757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.438758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.438846 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.439109 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.439109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.439110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.439252 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.439306 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.439630 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.439709 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 07:10:38.440023 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.430956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:43666)
0808 07:10:38.431155 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.431190 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.431662 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.431932 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.431935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.432176 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.432353 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.432357 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.433070 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.433071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.433888 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.433890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.434290 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.434292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.434433 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.434965 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.434973 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.436536 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.438448 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.438453 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.438455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.438679 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.438918 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.438919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.438919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.439040 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.439320 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.439325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.439492 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.439577 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.439757 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
I20260808 07:10:38.440701 7893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.54.125@KRBTEST.COM
I20260808 07:10:38.445104 7899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.435766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.30.54.125:39515)
0808 07:10:38.435916 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.435922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.435973 (+ 51us) server_negotiation.cc:407] Connection header received
0808 07:10:38.436043 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.436046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.436066 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.436151 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.436908 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.437778 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.438876 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.439278 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.439364 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.442008 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.442015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.442016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.442022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.443584 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.443990 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.443993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.443993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.444058 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.444343 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.444344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.444345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.444519 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.444673 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.444725 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.444807 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.125@KRBTEST.COM for oryx/127.30.54.125@KRBTEST.COM
I20260808 07:10:38.448364 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.440954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:39515 (local address 127.0.0.1:43974)
0808 07:10:38.441546 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.441557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.441610 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.441737 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.441738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.441883 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.441948 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.441950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.442890 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.442892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.443391 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.443393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.443865 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.443867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.443948 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.444383 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.444388 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.445774 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.447467 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.447468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.447468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.447633 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.447809 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.447809 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.447810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.447858 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.448124 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.448125 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.448160 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.448210 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.448282 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573}
I20260808 07:10:38.448508 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.441042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.30.54.125:39515)
0808 07:10:38.441201 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.441203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.441608 (+ 405us) server_negotiation.cc:407] Connection header received
0808 07:10:38.441665 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.441665 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.441673 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.441730 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.442017 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.442762 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.443475 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.443777 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.443825 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.445893 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.445894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.445894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.445900 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.447357 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.447698 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.447699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.447699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.447751 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.447965 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.447966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.447966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.448075 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.448116 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.448224 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.448327 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260808 07:10:38.449704 7893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: Bootstrap starting.
I20260808 07:10:38.450261 7893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:38.450485 7893 log.cc:824] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:38.451169 7893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: No bootstrap required, opened a new log
I20260808 07:10:38.452452 7893 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } }
I20260808 07:10:38.452564 7893 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:38.452586 7893 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4b9e938cd94ec1b42843f52bd9936e, State: Initialized, Role: FOLLOWER
I20260808 07:10:38.452702 7893 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } }
I20260808 07:10:38.452960 7904 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } } }
I20260808 07:10:38.453024 7893 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:38.453044 7904 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:10:38.454715 7917 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:10:38.454778 7917 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:10:38.456331 7796 master_runner.cc:428] Detected that this master ce4b9e938cd94ec1b42843f52bd9936e is joining an existing cluster
I20260808 07:10:38.456372 7796 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.54.125:39515
I20260808 07:10:38.464860 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.457563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:39515 (local address 127.0.0.1:43986)
0808 07:10:38.457683 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.457693 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.457751 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.457867 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.457868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.458043 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.458155 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.458157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.459760 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.459761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.460456 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.460458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.460911 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.460912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.460985 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.461332 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.461336 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.461647 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.463418 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.463419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.463420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.463600 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.463823 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.463824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.463824 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.463886 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.464171 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.464172 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.464555 (+ 383us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.464628 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.464734 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":6,"thread_start_us":72,"threads_started":1}
I20260808 07:10:38.464877 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.457593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.30.54.125:39515)
0808 07:10:38.457628 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.457630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.457759 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:10:38.457803 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.457803 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.457812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.457865 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.458891 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.459662 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.460538 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.460821 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.460904 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.461734 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.461735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.461735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.461741 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.463310 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.463690 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.463691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.463691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.463751 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.463955 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.463956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.463956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.464091 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.464151 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.464664 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.464789 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":18}
I20260808 07:10:38.464890 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.457377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.30.54.126:39687)
0808 07:10:38.457443 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.457445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.457574 (+ 129us) server_negotiation.cc:407] Connection header received
0808 07:10:38.457691 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.457693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.457708 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.457762 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.458239 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.459044 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.460212 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.460506 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.460584 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.461732 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.461734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.461735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.461743 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.463602 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.464009 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.464010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.464011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.464064 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.464291 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.464292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.464292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.464444 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.464528 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.464714 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.464795 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:10:38.465744 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.457297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:43680)
0808 07:10:38.457508 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.457520 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.457602 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.457810 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.457811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.458041 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.458155 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.458157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.459127 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.459128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.460106 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.460108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.460662 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.460664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.460734 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.461174 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.461180 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.461629 (+ 449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.463697 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.463698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.463698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.463943 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.464138 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.464139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.464139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.464215 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.464530 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.464531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.464577 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.465534 (+ 957us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.465641 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":8,"thread_start_us":96,"threads_started":1}
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.126@KRBTEST.COM for oryx/127.30.54.125@KRBTEST.COM
I20260808 07:10:38.474954 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.467921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43992 (local address 127.30.54.125:39515)
0808 07:10:38.467956 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.467958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.468081 (+ 123us) server_negotiation.cc:407] Connection header received
0808 07:10:38.468172 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.468173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.468183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.468239 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.468664 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.469109 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.469881 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.470084 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.470173 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.472352 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.472353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.472353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.472360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.473721 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.474117 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.474118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.474118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.474178 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.474382 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.474383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.474383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.474493 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.474569 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.474747 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.474851 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:10:38.474972 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.467839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:39515 (local address 127.0.0.1:43992)
0808 07:10:38.467951 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.468007 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.468106 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.468247 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.468250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.468527 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.468575 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.468577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.469201 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.469202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.469791 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.469793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.470178 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.470179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.470262 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.470729 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.470733 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.472252 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.473826 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.473834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.473836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.474037 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.474248 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.474248 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.474249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.474308 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.474563 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.474570 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.474622 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.474720 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.474854 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":52,"threads_started":1}
I20260808 07:10:38.475677 7137 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.54.125:39515: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e"
member_type: NON_VOTER
last_known_addr {
host: "127.30.54.125"
port: 39515
}
attrs {
promote: true
}
}
dest_uuid: "bec56ea19cb14faebd957b877001c29b"
cas_config_opid_index: -1
I20260808 07:10:38.476013 7137 consensus_queue.cc:237] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } }
I20260808 07:10:38.476504 7930 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ce4b9e938cd94ec1b42843f52bd9936e (127.30.54.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } } }
I20260808 07:10:38.476770 7857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:38.477583 7930 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.54.125:39515
I20260808 07:10:38.477692 7857 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Refusing update from remote peer bec56ea19cb14faebd957b877001c29b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 07:10:38.477676 7930 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bec56ea19cb14faebd957b877001c29b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } } }
I20260808 07:10:38.477739 7930 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:38.478104 7796 master.cc:537] Master@127.30.54.125:39515 shutting down...
I20260808 07:10:38.478062 7928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:10:38.478961 7118 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 07:10:38.479396 7118 consensus_peers.cc:487] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b -> Peer ce4b9e938cd94ec1b42843f52bd9936e (127.30.54.125:39515): Couldn't send request to peer ce4b9e938cd94ec1b42843f52bd9936e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:10:38.480558 7796 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:38.480643 7796 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:38.480715 7796 tablet_replica.cc:330] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: stopping tablet replica
I20260808 07:10:38.493696 7796 master.cc:552] Master@127.30.54.125:39515 shutdown complete.
I20260808 07:10:38.493737 7796 master_runner.cc:305] Clearing existing system tablet
I20260808 07:10:38.493887 7796 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.493932 7796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.494542 7796 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260808 07:10:38.495059 7936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.495183 7796 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.495239 7796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.495261 7796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal
uuid: "ce4b9e938cd94ec1b42843f52bd9936e"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "5ee1dc3a8eb799b402ce9d3b106bbb26"
server_key_iv: "78d84ba9e7c0e7f356b6ce311c8753d7"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.495900 7796 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:38.497491 7796 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:10:38.497550 7796 log.cc:1196] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 07:10:38.497754 7796 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: Deleting consensus metadata
I20260808 07:10:38.497879 7796 master_runner.cc:315] Copying system tablet from 127.30.54.126:39687
I20260808 07:10:38.498551 7796 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: tablet copy: Beginning tablet copy session from remote peer at address 127.30.54.126:39687
I20260808 07:10:38.504890 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.498883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:43696)
0808 07:10:38.499046 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.499055 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.499111 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.499249 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.499249 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.499398 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.499450 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.499452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.500384 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.500385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.500914 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.500915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.501376 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.501377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.501426 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.501916 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.501924 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.502209 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.503904 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.503905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.503905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.504067 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.504264 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.504264 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.504265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.504314 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.504596 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.504597 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.504641 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.504712 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.504813 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260808 07:10:38.504935 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.498947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43696 (local address 127.30.54.126:39687)
0808 07:10:38.498989 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.498991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.499127 (+ 136us) server_negotiation.cc:407] Connection header received
0808 07:10:38.499165 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.499166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.499179 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.499247 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.499518 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.500306 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.500997 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.501298 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.501372 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.502294 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.502295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.502295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.502303 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.503807 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.504137 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.504138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.504138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.504199 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.504374 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.504375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.504375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.504504 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.504576 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.504758 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.504838 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:38.505273 7166 tablet_copy_service.cc:140] P bec56ea19cb14faebd957b877001c29b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ce4b9e938cd94ec1b42843f52bd9936e ({username='oryx', principal='oryx/127.30.54.125@KRBTEST.COM'} at 127.0.0.1:43696)
I20260808 07:10:38.505371 7166 tablet_copy_service.cc:161] P bec56ea19cb14faebd957b877001c29b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ce4b9e938cd94ec1b42843f52bd9936e at {username='oryx', principal='oryx/127.30.54.125@KRBTEST.COM'} at 127.0.0.1:43696: session id = ce4b9e938cd94ec1b42843f52bd9936e-00000000000000000000000000000000
I20260808 07:10:38.505980 7166 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:10:38.506554 7796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:10:38.507498 7796 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: tablet copy: Starting download of 0 data blocks...
I20260808 07:10:38.507660 7796 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: tablet copy: Starting download of 1 WAL segments...
I20260808 07:10:38.508498 7796 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:10:38.509270 7166 tablet_copy_service.cc:342] P bec56ea19cb14faebd957b877001c29b: Request end of tablet copy session ce4b9e938cd94ec1b42843f52bd9936e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.54.125@KRBTEST.COM'} at 127.0.0.1:43696
I20260808 07:10:38.509330 7166 tablet_copy_service.cc:433] P bec56ea19cb14faebd957b877001c29b: ending tablet copy session ce4b9e938cd94ec1b42843f52bd9936e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ce4b9e938cd94ec1b42843f52bd9936e
I20260808 07:10:38.512059 7796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:38.513188 7946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.513293 7947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.513315 7949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.513568 7796 server_base.cc:1034] running on GCE node
I20260808 07:10:38.513674 7796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:38.513808 7796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:38.514930 7796 hybrid_clock.cc:648] HybridClock initialized: now 1786173038514919 us; error 25 us; skew 500 ppm
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:38.516369 7796 init.cc:377] Logged in from keytab as oryx/127.30.54.125@KRBTEST.COM (short username oryx)
I20260808 07:10:38.516674 7796 webserver.cc:460] Webserver started at http://127.30.54.125:41601/ using document root <none> and password file <none>
I20260808 07:10:38.516791 7796 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.516848 7796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.517446 7796 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.517972 7957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.518126 7796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.518184 7796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal
uuid: "ce4b9e938cd94ec1b42843f52bd9936e"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "5ee1dc3a8eb799b402ce9d3b106bbb26"
server_key_iv: "78d84ba9e7c0e7f356b6ce311c8753d7"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.518236 7796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.539001 7796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:38.540587 7796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:38.543915 7796 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.125:39515
I20260808 07:10:38.543987 8030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.125:39515 every 8 connection(s)
I20260808 07:10:38.544260 7796 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
I20260808 07:10:38.544934 8031 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:10:38.545398 8031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: Bootstrap starting.
I20260808 07:10:38.548009 8031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:10:38.548251 8031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: Bootstrap complete.
I20260808 07:10:38.548470 8031 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } }
I20260808 07:10:38.548559 8031 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ce4b9e938cd94ec1b42843f52bd9936e, State: Initialized, Role: LEARNER
I20260808 07:10:38.548604 8031 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } }
I20260808 07:10:38.548815 8031 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:38.548808 8036 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } } }
I20260808 07:10:38.548936 8036 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: This master's current role is: LEARNER
I20260808 07:10:38.550302 8053 catalog_manager.cc:1259] Loaded cluster ID: 8bc5edd0134b4405b0518f7483db023f
I20260808 07:10:38.550354 8053 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: loading cluster ID for follower catalog manager: success
I20260808 07:10:38.551504 8053 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: acquiring CA information for follower catalog manager: success
I20260808 07:10:38.552012 8053 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:10:38.558656 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.546842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.30.54.125:39515)
0808 07:10:38.550137 (+ 3295us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.550139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.550149 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:38.550173 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.550174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.550187 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.550996 (+ 809us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.551030 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.552075 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.552819 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.553049 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.553129 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.554341 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.554342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.554343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.554350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.556984 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.557541 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.557542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.557542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.557622 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.557949 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.557951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.557951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.558105 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.558219 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.558439 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.558548 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3273,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for oryx/127.30.54.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.124.keytab.
Entry for principal oryx/127.30.54.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.124.keytab.
I20260808 07:10:38.595005 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.124:42045
--webserver_interface=127.30.54.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.124.keytab
--principal=oryx/127.30.54.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:38.682379 8057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:38.682574 8057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:38.682595 8057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:38.683769 8057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:38.683812 8057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:38.683826 8057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:38.683840 8057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:38.683856 8057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:38.685289 8057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.124.keytab
--principal=oryx/127.30.54.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.124:42045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.54.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:38.685487 8057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:38.685645 8057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:38.690378 8068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.690405 8065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.690409 8066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.690837 8057 server_base.cc:1034] running on GCE node
I20260808 07:10:38.691075 8057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:38.691386 8057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:38.692556 8057 hybrid_clock.cc:648] HybridClock initialized: now 1786173038692546 us; error 39 us; skew 500 ppm
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:38.695065 8057 init.cc:377] Logged in from keytab as oryx/127.30.54.124@KRBTEST.COM (short username oryx)
I20260808 07:10:38.695495 8057 webserver.cc:460] Webserver started at http://127.30.54.124:39569/ using document root <none> and password file <none>
I20260808 07:10:38.695730 8057 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.695799 8057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.695955 8057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:38.696980 8057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data/instance:
uuid: "8646d7de064b4751b36bea7e3f35e304"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "75ae43ce9ae9439c01aff649450cf728"
server_key_iv: "f3e10563ab25b38f56addd342bef4255"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.697351 8057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal/instance:
uuid: "8646d7de064b4751b36bea7e3f35e304"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "75ae43ce9ae9439c01aff649450cf728"
server_key_iv: "f3e10563ab25b38f56addd342bef4255"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.698801 8057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:38.699699 8081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.699872 8057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:38.699980 8057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal
uuid: "8646d7de064b4751b36bea7e3f35e304"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "75ae43ce9ae9439c01aff649450cf728"
server_key_iv: "f3e10563ab25b38f56addd342bef4255"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.700054 8057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.732391 8057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:38.734747 8057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:38.734942 8057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:38.739445 8057 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.124:42045
I20260808 07:10:38.739483 8145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.124:42045 every 8 connection(s)
I20260808 07:10:38.739856 8057 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
I20260808 07:10:38.740770 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 8057
I20260808 07:10:38.741083 8146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:38.743554 8146 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.54.124@KRBTEST.COM
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.124@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:10:38.751986 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.741528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.30.54.124:42045)
0808 07:10:38.741759 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.741766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.741790 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:10:38.742200 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.742206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.742250 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.742372 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.743180 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.744193 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.744961 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.745376 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.745518 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.748140 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.748148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.748148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.748155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.750305 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.750713 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.750715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.750715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.750793 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.751062 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.751062 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.751063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.751349 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.751545 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.751667 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.751759 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":113,"threads_started":1}
I20260808 07:10:38.752241 8150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.744121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:43708)
0808 07:10:38.744363 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.744391 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.744487 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.744685 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.744688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.744932 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.745001 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.745002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.745661 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.745662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.746238 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.746240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.746623 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.746625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.746688 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.747131 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.747141 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.749018 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.750988 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.750992 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.750994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.751253 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.751469 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.751469 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.751470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.751533 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.751887 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.751898 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.751943 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.752037 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.752143 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":101,"threads_started":1}
I20260808 07:10:38.752270 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.744260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43708 (local address 127.30.54.126:39687)
0808 07:10:38.744306 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.744308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.744460 (+ 152us) server_negotiation.cc:407] Connection header received
0808 07:10:38.744562 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.744563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.744574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.744656 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.745084 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.745562 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.746327 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.746521 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.746595 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.749112 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.749113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.749114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.749121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.750917 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.751328 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.751328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.751329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.751376 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.751602 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.751603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.751604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.751743 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.751896 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.752098 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.752181 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:10:38.752959 8146 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.124@KRBTEST.COM for oryx/127.30.54.125@KRBTEST.COM
I20260808 07:10:38.758792 8150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.753209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:39515 (local address 127.0.0.1:44006)
0808 07:10:38.753256 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.753267 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.753301 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.753429 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.753430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.753534 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.753575 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.753576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.754163 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.754163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.754626 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.754627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.755007 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.755008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.755059 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.755357 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.755360 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.756413 (+ 1053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.757998 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.757999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.757999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.758150 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.758323 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.758324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.758324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.758372 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.758583 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.758583 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.758614 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.758662 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.758737 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:10:38.758797 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.753290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.30.54.125:39515)
0808 07:10:38.753331 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.753333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.753341 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:10:38.753364 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.753365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.753381 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.753428 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.753635 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.754095 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.754692 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.754951 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.755008 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.756489 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.756490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.756491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.756497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.757920 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.758214 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.758215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.758215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.758265 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.758423 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.758424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.758424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.758530 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.758579 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.758677 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.758739 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:10:38.759259 8146 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 42045 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.124@KRBTEST.COM for oryx/127.30.54.124@KRBTEST.COM
I20260808 07:10:38.765339 8150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.759503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:42045 (local address 127.0.0.1:56684)
0808 07:10:38.759551 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.759561 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.759598 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.759695 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.759696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.759801 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.759826 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.759827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.760395 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.760396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.760863 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.760864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.761248 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.761249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.761296 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.761654 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.761658 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.762738 (+ 1080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.764442 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.764443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.764443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.764620 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.764852 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.764853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.764853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.764907 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.765128 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.765129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.765161 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.765211 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.765283 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:10:38.765419 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.759551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56684 (local address 127.30.54.124:42045)
0808 07:10:38.759610 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.759611 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.759617 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:10:38.759647 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.759647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.759654 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.759692 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.759879 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.760308 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.760963 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.761170 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.761233 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.762867 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.762868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.762869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.762876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.764347 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.764712 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.764714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.764714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.764774 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.764968 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.764969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.764969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.765073 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.765126 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.765230 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.765342 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":41}
I20260808 07:10:38.767071 8146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: Bootstrap starting.
I20260808 07:10:38.767792 8146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:38.768082 8146 log.cc:824] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:38.768910 8146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: No bootstrap required, opened a new log
I20260808 07:10:38.770308 8146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 42045 } }
I20260808 07:10:38.770488 8146 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:38.770538 8146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8646d7de064b4751b36bea7e3f35e304, State: Initialized, Role: FOLLOWER
I20260808 07:10:38.770685 8146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 42045 } }
I20260808 07:10:38.770999 8152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 42045 } } }
I20260808 07:10:38.771068 8146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:38.771097 8152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:10:38.772797 8171 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:10:38.772871 8171 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:10:38.775665 8057 master_runner.cc:428] Detected that this master 8646d7de064b4751b36bea7e3f35e304 is joining an existing cluster
I20260808 07:10:38.775717 8057 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.54.124:42045
I20260808 07:10:38.784353 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.776817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:39515 (local address 127.0.0.1:44016)
0808 07:10:38.777184 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.777193 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.777268 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.777448 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.777449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.777830 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.777915 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.777916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.778888 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.778888 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.779854 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.779863 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.780639 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.780639 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.780689 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.781197 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.781201 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.781609 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.783422 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.783423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.783423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.783588 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.783796 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.783797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.783797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.783847 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.784114 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.784115 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.784160 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.784211 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.784283 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":42,"thread_start_us":115,"threads_started":1}
I20260808 07:10:38.784452 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.776752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44016 (local address 127.30.54.125:39515)
0808 07:10:38.776793 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.776795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.777243 (+ 448us) server_negotiation.cc:407] Connection header received
0808 07:10:38.777340 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.777341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.777353 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.777383 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.778009 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.778760 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.779959 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.780257 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.780296 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.781705 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.781706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.781707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.781714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.783342 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.783662 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.783664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.783664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.783729 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.783910 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.783911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.783911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.784048 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.784121 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.784283 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.784362 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:10:38.802750 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.777073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:42045 (local address 127.0.0.1:56690)
0808 07:10:38.777187 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.777196 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.777278 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.777431 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.777432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.777743 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.777824 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.777826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.778698 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.778699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.779614 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.779618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.780093 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.780093 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.780205 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.780657 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.780662 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.799167 (+ 18505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.801485 (+ 2318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.801486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.801486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.801705 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.801917 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.801917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.801918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.801987 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.802340 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.802341 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.802400 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.802477 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.802585 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260808 07:10:38.803208 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.776767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.30.54.126:39687)
0808 07:10:38.776818 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.776820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.777099 (+ 279us) server_negotiation.cc:407] Connection header received
0808 07:10:38.777442 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.777443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.777453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.777485 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.778010 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.778777 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.779959 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.780257 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.780296 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.799887 (+ 19591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.799890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.799890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.799900 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.801736 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.802132 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.802133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.802133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.802188 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.802395 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.802396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.802396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.802537 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.802606 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.802999 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.803117 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:10:38.803246 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.777131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56690 (local address 127.30.54.124:42045)
0808 07:10:38.777179 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.777181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.777244 (+ 63us) server_negotiation.cc:407] Connection header received
0808 07:10:38.777344 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.777345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.777353 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.777379 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.777908 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.778613 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.779710 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.779974 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.780016 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.799308 (+ 19292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.799310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.799311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.799319 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.801391 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.801790 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.801791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.801792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.801857 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.802061 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.802062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.802062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.802227 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.802296 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.802502 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.802652 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":35}
I20260808 07:10:38.803505 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 7226
I20260808 07:10:38.803623 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.776706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:43724)
0808 07:10:38.777020 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.777029 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.777339 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.777549 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.777550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.777830 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.777914 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.777916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.778911 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.778911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.779854 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.779856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.780389 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.780390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.780455 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.799382 (+ 18927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.799386 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.799750 (+ 364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.801838 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.801838 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.801839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.802030 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.802276 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.802276 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.802277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.802338 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.802776 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.802777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.802852 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.803361 (+ 509us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.803503 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":36,"thread_start_us":213,"threads_started":1}
I20260808 07:10:38.841356 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:38663
--local_ip_for_outbound_sockets=127.30.54.65
--tserver_master_addrs=127.30.54.126:39687
--webserver_port=41513
--webserver_interface=127.30.54.65
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab
--principal=oryx/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.54.126:39687,127.30.54.125:39515
--tserver_master_addrs=127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.126@KRBTEST.COM for oryx/127.30.54.124@KRBTEST.COM
I20260808 07:10:38.847178 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.838536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:42045 (local address 127.0.0.1:56698)
0808 07:10:38.838598 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.838609 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.838654 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.838806 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.838806 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.839100 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.839165 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.839168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.840103 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.840104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.841110 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.841113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.841678 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.841679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.841748 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.842178 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.842182 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.843775 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.845736 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.845737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.845738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.846011 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.846242 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.846243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.846243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.846321 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.846631 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.846632 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.846693 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.846866 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.846996 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:10:38.847661 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.838599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56698 (local address 127.30.54.124:42045)
0808 07:10:38.838642 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.838644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.838654 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:38.838724 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.838725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.838736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.838799 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.839263 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.840000 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.841215 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.841539 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.841616 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.843878 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.843880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.843880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.843888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.845633 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.846097 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.846099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.846099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.846159 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.846404 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.846405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.846406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.846546 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.846618 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.847410 (+ 792us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.847510 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:10:38.848286 7137 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.54.124:42045: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8646d7de064b4751b36bea7e3f35e304"
member_type: NON_VOTER
last_known_addr {
host: "127.30.54.124"
port: 42045
}
attrs {
promote: true
}
}
dest_uuid: "bec56ea19cb14faebd957b877001c29b"
cas_config_opid_index: 5
I20260808 07:10:38.848594 7137 consensus_queue.cc:237] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:10:38.849050 7930 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8646d7de064b4751b36bea7e3f35e304 (127.30.54.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.849648 8112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:38.850181 7930 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.54.124:42045
I20260808 07:10:38.850268 7930 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bec56ea19cb14faebd957b877001c29b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.850335 7930 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:38.850754 8112 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [term 1 FOLLOWER]: Refusing update from remote peer bec56ea19cb14faebd957b877001c29b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:10:38.851003 8057 master.cc:537] Master@127.30.54.124:42045 shutting down...
I20260808 07:10:38.851078 7928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:10:38.851745 7117 consensus_peers.cc:487] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b -> Peer 8646d7de064b4751b36bea7e3f35e304 (127.30.54.124:42045): Couldn't send request to peer 8646d7de064b4751b36bea7e3f35e304. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:10:38.855192 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.849266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:39515 (local address 127.0.0.1:44026)
0808 07:10:38.849568 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.849579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.849650 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.849908 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.849909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.849910 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:38.849947 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.849949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.851046 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.851046 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.852724 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.852728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.854678 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.854679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.854766 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.854795 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.855035 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.855089 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286}
I20260808 07:10:38.855305 8057 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:38.855397 8057 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:38.855440 8057 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: stopping tablet replica
I20260808 07:10:38.855688 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.849340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.30.54.125:39515)
0808 07:10:38.849393 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.849395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.849624 (+ 229us) server_negotiation.cc:407] Connection header received
0808 07:10:38.849725 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.849726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.849752 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.849795 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:38.850035 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.850861 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.853946 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.854576 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.854656 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.854727 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.855533 (+ 806us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.855590 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:10:38.856385 7995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 LEARNER]: Refusing update from remote peer bec56ea19cb14faebd957b877001c29b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:10:38.856694 8188 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 07:10:38.857432 7995 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8646d7de064b4751b36bea7e3f35e304 (127.30.54.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.858781 8036 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bec56ea19cb14faebd957b877001c29b. Latest consensus state: current_term: 1 leader_uuid: "bec56ea19cb14faebd957b877001c29b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.858938 8036 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: This master's current role is: LEARNER
I20260808 07:10:38.858800 8191 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bec56ea19cb14faebd957b877001c29b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: NON_VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: true } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.859107 8191 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: This master's current role is: LEARNER
I20260808 07:10:38.859623 8188 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b: attempting to promote NON_VOTER ce4b9e938cd94ec1b42843f52bd9936e to VOTER
I20260808 07:10:38.859881 8188 consensus_queue.cc:237] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:10:38.860553 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.860415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:42045 (local address 127.0.0.1:56706)
0808 07:10:38.860474 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.860523 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:42045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:10:38.860579 7995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 LEARNER]: Refusing update from remote peer bec56ea19cb14faebd957b877001c29b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 07:10:38.860915 7928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 07:10:38.860976 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.860794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:42045 (local address 127.0.0.1:56722)
0808 07:10:38.860888 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.860913 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:42045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260808 07:10:38.861038 7117 consensus_peers.cc:487] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b -> Peer 8646d7de064b4751b36bea7e3f35e304 (127.30.54.124:42045): Couldn't send request to peer 8646d7de064b4751b36bea7e3f35e304. Status: Network error: Client connection negotiation failed: client connection to 127.30.54.124:42045: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 07:10:38.861537 7928 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ce4b9e938cd94ec1b42843f52bd9936e (127.30.54.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.862432 7928 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bec56ea19cb14faebd957b877001c29b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.862507 7928 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:38.862564 7928 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bec56ea19cb14faebd957b877001c29b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.862612 7928 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:38.862936 7995 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ce4b9e938cd94ec1b42843f52bd9936e (127.30.54.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.863878 8191 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bec56ea19cb14faebd957b877001c29b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.863969 8191 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:10:38.868804 8057 master.cc:552] Master@127.30.54.124:42045 shutdown complete.
I20260808 07:10:38.868858 8057 master_runner.cc:305] Clearing existing system tablet
I20260808 07:10:38.868991 8057 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.869043 8057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.869830 8057 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:38.870395 8193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.870673 8057 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.870748 8057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:10:38.870775 8057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal
uuid: "8646d7de064b4751b36bea7e3f35e304"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "75ae43ce9ae9439c01aff649450cf728"
server_key_iv: "f3e10563ab25b38f56addd342bef4255"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.871682 8057 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:38.873207 8057 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:10:38.873293 8057 log.cc:1196] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 07:10:38.873513 8057 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: Deleting consensus metadata
I20260808 07:10:38.873651 8057 master_runner.cc:315] Copying system tablet from 127.30.54.126:39687
I20260808 07:10:38.874279 8057 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: tablet copy: Beginning tablet copy session from remote peer at address 127.30.54.126:39687
I20260808 07:10:38.882056 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.874563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:43736)
0808 07:10:38.874711 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:38.874723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:38.874789 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:38.874956 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:38.874957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:38.875169 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:38.875232 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.875234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.876275 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.876277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.877182 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:38.877186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.877762 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.877763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:38.877834 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.878369 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:38.878375 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:38.878786 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:38.880826 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.880827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.880828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.881073 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.881312 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:38.881313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:38.881313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:38.881387 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:38.881695 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:38.881696 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:38.881759 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 07:10:38.881842 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:10:38.881949 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260808 07:10:38.882069 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.874749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.30.54.126:39687)
0808 07:10:38.874789 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.874791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.874832 (+ 41us) server_negotiation.cc:407] Connection header received
0808 07:10:38.874864 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.874865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.874877 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.874949 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.875329 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.876179 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.877330 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.877655 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.877736 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.878921 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.878922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.878923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:38.878931 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.880729 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.881161 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.881162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.881163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.881231 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:38.881465 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:38.881466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:38.881466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:38.881611 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.881687 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.881888 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.881974 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:38.882692 7166 tablet_copy_service.cc:140] P bec56ea19cb14faebd957b877001c29b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8646d7de064b4751b36bea7e3f35e304 ({username='oryx', principal='oryx/127.30.54.124@KRBTEST.COM'} at 127.0.0.1:43736)
I20260808 07:10:38.882769 7166 tablet_copy_service.cc:161] P bec56ea19cb14faebd957b877001c29b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8646d7de064b4751b36bea7e3f35e304 at {username='oryx', principal='oryx/127.30.54.124@KRBTEST.COM'} at 127.0.0.1:43736: session id = 8646d7de064b4751b36bea7e3f35e304-00000000000000000000000000000000
I20260808 07:10:38.883440 7166 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:10:38.883980 8057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:10:38.885335 8057 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: tablet copy: Starting download of 0 data blocks...
I20260808 07:10:38.885500 8057 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: tablet copy: Starting download of 1 WAL segments...
I20260808 07:10:38.886411 8057 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:10:38.887491 7166 tablet_copy_service.cc:342] P bec56ea19cb14faebd957b877001c29b: Request end of tablet copy session 8646d7de064b4751b36bea7e3f35e304-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.54.124@KRBTEST.COM'} at 127.0.0.1:43736
I20260808 07:10:38.887557 7166 tablet_copy_service.cc:433] P bec56ea19cb14faebd957b877001c29b: ending tablet copy session 8646d7de064b4751b36bea7e3f35e304-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8646d7de064b4751b36bea7e3f35e304
I20260808 07:10:38.890964 8057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:38.892230 8202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.895908 8204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.896032 8057 server_base.cc:1034] running on GCE node
W20260808 07:10:38.895939 8206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.896432 8057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:38.896588 8057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:38.897730 8057 hybrid_clock.cc:648] HybridClock initialized: now 1786173038897719 us; error 28 us; skew 500 ppm
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:38.899710 8057 init.cc:377] Logged in from keytab as oryx/127.30.54.124@KRBTEST.COM (short username oryx)
I20260808 07:10:38.900008 8057 webserver.cc:460] Webserver started at http://127.30.54.124:41931/ using document root <none> and password file <none>
I20260808 07:10:38.900131 8057 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.900161 8057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.900923 8057 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:38.901547 8214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.901714 8057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.901767 8057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal
uuid: "8646d7de064b4751b36bea7e3f35e304"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "75ae43ce9ae9439c01aff649450cf728"
server_key_iv: "f3e10563ab25b38f56addd342bef4255"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.901829 8057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.953101 8057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:38.954870 8057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:38.958387 8057 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.124:42045
I20260808 07:10:38.958945 8057 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
I20260808 07:10:38.959311 8291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.124:42045 every 8 connection(s)
I20260808 07:10:38.959892 8292 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:10:38.960332 8292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: Bootstrap starting.
I20260808 07:10:38.962738 8292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:10:38.962985 8292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: Bootstrap complete.
I20260808 07:10:38.963186 8292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:10:38.963253 8292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8646d7de064b4751b36bea7e3f35e304, State: Initialized, Role: LEARNER
I20260808 07:10:38.963313 8292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:10:38.963487 8292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:38.963522 8299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } } }
I20260808 07:10:38.963662 8299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304 [sys.catalog]: This master's current role is: LEARNER
W20260808 07:10:38.964661 8187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:38.964951 8187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:38.965003 8187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 07:10:38.964984 8312 catalog_manager.cc:1259] Loaded cluster ID: 8bc5edd0134b4405b0518f7483db023f
I20260808 07:10:38.965034 8312 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: loading cluster ID for follower catalog manager: success
I20260808 07:10:38.965938 8312 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: acquiring CA information for follower catalog manager: success
I20260808 07:10:38.966202 8312 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8646d7de064b4751b36bea7e3f35e304: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 07:10:38.966732 8187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:38.966801 8187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:38.966876 8187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:10:38.968592 8187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab
--principal=oryx/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:38663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=41513
--tserver_master_addrs=127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:38.968858 8187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:38.969102 8187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:38.974402 8320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.974431 8317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.974386 8316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.974608 8187 server_base.cc:1034] running on GCE node
I20260808 07:10:38.974877 8187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:38.975143 8187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:38.976377 8187 hybrid_clock.cc:648] HybridClock initialized: now 1786173038976352 us; error 36 us; skew 500 ppm
Aug 08 07:10:38 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:38.979135 8187 init.cc:377] Logged in from keytab as oryx/127.30.54.65@KRBTEST.COM (short username oryx)
I20260808 07:10:38.979651 8187 webserver.cc:460] Webserver started at http://127.30.54.65:41513/ using document root <none> and password file <none>
I20260808 07:10:38.979859 8187 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.979931 8187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.981285 8187 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:38.981895 8330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.982103 8187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:38.982197 8187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "762dda4727d845a5bb56adb9828d3308"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-znh6"
server_key: "d7b58158ab80505527b194027e266e13"
server_key_iv: "17cdc863390205974d485b2537d125db"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.982599 8187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.994714 8187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:38.996608 8187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:38.996762 8187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:38.997007 8187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:38.997346 8187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:38.997414 8187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.997454 8187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:38.997490 8187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:39.004191 8187 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:38663
I20260808 07:10:39.004245 8514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:38663 every 8 connection(s)
I20260808 07:10:39.004599 8187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:39.008929 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 8187
I20260808 07:10:39.009091 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 7415
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.125@KRBTEST.COM
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.124@KRBTEST.COM
I20260808 07:10:39.020738 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.005326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:52029 (local address 127.30.54.125:39515)
0808 07:10:39.005377 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.005379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.005632 (+ 253us) server_negotiation.cc:407] Connection header received
0808 07:10:39.006808 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.006810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.006824 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.006905 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.007736 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.008283 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.012229 (+ 3946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.012571 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.012626 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.015896 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.015898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.015899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.015907 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.018857 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.019660 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.019661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.019661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.019721 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.019914 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.019914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.019915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.020061 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.020102 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.020494 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.020591 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:10:39.020797 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.005314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:37521 (local address 127.30.54.126:39687)
0808 07:10:39.005398 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.005400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.005635 (+ 235us) server_negotiation.cc:407] Connection header received
0808 07:10:39.006790 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.006792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.006805 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.006887 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.007734 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.008524 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.009738 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.009977 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.010015 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.015230 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.015232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.015232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.015240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.018337 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.019398 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.019399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.019400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.019482 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.019887 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.019889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.019889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.020073 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.020130 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.020526 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.020639 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:10:39.021546 8523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.005326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:39515 (local address 127.30.54.65:52029)
0808 07:10:39.005506 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.005550 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.006676 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.006993 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.006997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.007390 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:39.007619 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.007624 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.008407 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.008408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.012108 (+ 3700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.012110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.012724 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.012725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.012795 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.013623 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:39.013629 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:39.015799 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:39.019392 (+ 3593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.019393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.019393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.019577 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.019792 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.019793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.019793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.019851 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.020142 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:39.020149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:39.020334 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.021056 (+ 722us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.021285 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":127,"thread_start_us":103,"threads_started":1}
I20260808 07:10:39.021553 8524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.005327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:37521)
0808 07:10:39.005508 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.005549 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.006676 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.006992 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.006996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.007431 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:39.007621 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.007627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.008602 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.008603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.009638 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.009641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.010096 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.010097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.010217 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.011952 (+ 1735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:39.011963 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:39.015119 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:39.018966 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.018975 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.018977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.019299 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.019585 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.019586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.019586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.019771 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.020190 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:39.020191 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:39.020337 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.020921 (+ 584us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.021197 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":45,"thread_start_us":111,"threads_started":1}
I20260808 07:10:39.022156 8518 heartbeater.cc:347] Connected to a master server at 127.30.54.125:39515
I20260808 07:10:39.022285 8518 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:39.022558 8518 heartbeater.cc:511] Master 127.30.54.125:39515 requested a full tablet report, sending...
I20260808 07:10:39.023221 7979 ts_manager.cc:194] Registered new tserver with Master: 762dda4727d845a5bb56adb9828d3308 (127.30.54.65:38663)
I20260808 07:10:39.025171 8517 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39687
I20260808 07:10:39.025238 8517 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:39.025334 8517 heartbeater.cc:511] Master 127.30.54.126:39687 requested a full tablet report, sending...
I20260808 07:10:39.025666 7137 ts_manager.cc:194] Re-registered known tserver with Master: 762dda4727d845a5bb56adb9828d3308 (127.30.54.65:38663)
I20260808 07:10:39.026213 7137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:37521
I20260808 07:10:39.028155 8526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.005644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:42045 (local address 127.30.54.65:51301)
0808 07:10:39.005775 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.005785 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.006679 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.006992 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.006996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.007420 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:39.007621 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.007626 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.008848 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.008849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.009725 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.009728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.010397 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.010398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.010478 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.016083 (+ 5605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:39.016089 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:39.024342 (+ 8253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:39.027165 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.027166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.027166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.027331 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.027562 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.027562 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.027562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.027605 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.027892 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:39.027893 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:39.027926 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.027981 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.028073 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":4,"thread_start_us":64,"threads_started":1}
I20260808 07:10:39.028292 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.005773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:51301 (local address 127.30.54.124:42045)
0808 07:10:39.005914 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.005916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.005927 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:39.006871 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.006872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.006887 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.006971 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.007733 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.008716 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.009831 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.010260 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.010334 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.024454 (+ 14120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.024457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.024457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.024467 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.027082 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.027414 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.027416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.027417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.027491 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.027646 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.027648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.027648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.027815 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.027895 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.028012 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.028118 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260808 07:10:39.028687 8515 heartbeater.cc:347] Connected to a master server at 127.30.54.124:42045
I20260808 07:10:39.028740 8515 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:39.028817 8515 heartbeater.cc:511] Master 127.30.54.124:42045 requested a full tablet report, sending...
I20260808 07:10:39.029227 8235 ts_manager.cc:194] Registered new tserver with Master: 762dda4727d845a5bb56adb9828d3308 (127.30.54.65:38663)
I20260808 07:10:39.030202 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:45907
--local_ip_for_outbound_sockets=127.30.54.66
--tserver_master_addrs=127.30.54.126:39687
--webserver_port=39559
--webserver_interface=127.30.54.66
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab
--principal=oryx/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.54.126:39687,127.30.54.125:39515
--tserver_master_addrs=127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:39.113345 8530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:39.113526 8530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:39.113548 8530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:39.114650 8530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:39.114693 8530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:39.114719 8530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:10:39.116205 8530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab
--principal=oryx/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:45907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=39559
--tserver_master_addrs=127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:39.116436 8530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:39.116673 8530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:39.121869 8539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:39.121870 8530 server_base.cc:1034] running on GCE node
W20260808 07:10:39.121836 8535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:39.121832 8536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:39.122311 8530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:39.122566 8530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:39.123726 8530 hybrid_clock.cc:648] HybridClock initialized: now 1786173039123711 us; error 31 us; skew 500 ppm
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173039, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:39.126384 8530 init.cc:377] Logged in from keytab as oryx/127.30.54.66@KRBTEST.COM (short username oryx)
I20260808 07:10:39.126919 8530 webserver.cc:460] Webserver started at http://127.30.54.66:39559/ using document root <none> and password file <none>
I20260808 07:10:39.127188 8530 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:39.127264 8530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:39.128549 8530 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:39.129122 8552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:39.129316 8530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:39.129407 8530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "933fc011f83a4f16a77ddea9e4d2936d"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "230f6b57491245ee2c36b35f8d592c3b"
server_key_iv: "5349211570e3fff1e9ffc4605b283a0d"
server_key_version: "encryptionkey@0"
I20260808 07:10:39.129685 8530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:39.155265 8530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:39.157552 8530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:39.157662 8530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:39.157848 8530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:39.158175 8530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:39.158211 8530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:39.158233 8530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:39.158282 8530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:39.164166 8530 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:45907
I20260808 07:10:39.164227 8711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:45907 every 8 connection(s)
I20260808 07:10:39.164618 8530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:39.165884 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 8530
I20260808 07:10:39.166057 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 7609
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173039, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.125@KRBTEST.COM
I20260808 07:10:39.177772 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.165403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:54171 (local address 127.30.54.125:39515)
0808 07:10:39.165480 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.165482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.165711 (+ 229us) server_negotiation.cc:407] Connection header received
0808 07:10:39.166378 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.166380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.166392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.166421 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.168172 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.168938 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.170249 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.170500 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.170553 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.173906 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.173907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.173917 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.173925 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.176077 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.176552 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.176554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.176554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.176632 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.176933 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.176934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.176935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.177119 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.177208 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.177489 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.177615 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:10:39.177968 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.165392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:39515 (local address 127.30.54.66:54171)
0808 07:10:39.165571 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.165612 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.166268 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.166608 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.166613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.167652 (+ 1039us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:39.168038 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.168046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.169042 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.169043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.170162 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.170164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.170809 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.170824 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.170958 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.172159 (+ 1201us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:39.172168 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:39.173807 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:39.176167 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.176174 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.176176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.176465 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.176711 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.176712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.176712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.176837 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.177197 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:39.177202 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:39.177343 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.177440 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.177606 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260808 07:10:39.183172 8719 heartbeater.cc:347] Connected to a master server at 127.30.54.125:39515
I20260808 07:10:39.183281 8719 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:39.183481 8719 heartbeater.cc:511] Master 127.30.54.125:39515 requested a full tablet report, sending...
I20260808 07:10:39.183962 7979 ts_manager.cc:194] Registered new tserver with Master: 933fc011f83a4f16a77ddea9e4d2936d (127.30.54.66:45907)
I20260808 07:10:39.186439 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:42685
--local_ip_for_outbound_sockets=127.30.54.67
--tserver_master_addrs=127.30.54.126:39687
--webserver_port=35913
--webserver_interface=127.30.54.67
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab
--principal=oryx/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.54.126:39687,127.30.54.125:39515
--tserver_master_addrs=127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173039, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.124@KRBTEST.COM
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173039, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:10:39.190285 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.165489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:46757 (local address 127.30.54.124:42045)
0808 07:10:39.165540 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.165543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.165688 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:10:39.182344 (+ 16656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.182346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.182359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.182500 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.183001 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.183754 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.184752 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.185119 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.185192 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.187380 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.187382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.187382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.187390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.188986 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.189376 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.189377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.189378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.189438 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.189670 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.189671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.189671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.189823 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.189896 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.190083 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.190172 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:10:39.190302 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.165394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:42045 (local address 127.30.54.66:46757)
0808 07:10:39.165593 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.165617 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.182262 (+ 16645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.182546 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.182547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.182807 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:39.182897 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.182899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.183842 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.183843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.184664 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.184666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.185206 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.185207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.185279 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.185762 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:39.185769 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:39.187284 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:39.189051 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.189052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.189052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.189273 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.189510 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.189511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.189512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.189582 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.189901 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:39.189902 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:39.189951 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.190019 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.190134 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260808 07:10:39.191025 8714 heartbeater.cc:347] Connected to a master server at 127.30.54.124:42045
I20260808 07:10:39.191110 8714 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:39.191200 8714 heartbeater.cc:511] Master 127.30.54.124:42045 requested a full tablet report, sending...
I20260808 07:10:39.191522 8235 ts_manager.cc:194] Registered new tserver with Master: 933fc011f83a4f16a77ddea9e4d2936d (127.30.54.66:45907)
I20260808 07:10:39.191735 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.165886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:52483 (local address 127.30.54.126:39687)
0808 07:10:39.165940 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.165943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.165952 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:10:39.182276 (+ 16324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.182277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.182291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.182345 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.183783 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.184552 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.185446 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.185844 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.185914 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.188559 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.188561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.188561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.188570 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.190304 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.190725 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.190726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.190727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.190794 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.191176 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.191177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.191177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.191312 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.191375 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.191549 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.191634 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:10:39.191776 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.165698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:52483)
0808 07:10:39.165823 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.165832 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.182174 (+ 16342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.182394 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.182395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.183093 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:39.183680 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.183682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.184756 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.184757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.185347 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.185349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.186012 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.186013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.186080 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.186923 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:39.186928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:39.188458 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:39.190397 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.190399 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.190399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.190626 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.190995 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.190996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.190996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.191050 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.191380 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:39.191381 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:39.191426 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.191488 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.191602 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260808 07:10:39.192293 8715 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39687
I20260808 07:10:39.192358 8715 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:39.192431 8715 heartbeater.cc:511] Master 127.30.54.126:39687 requested a full tablet report, sending...
I20260808 07:10:39.192668 7137 ts_manager.cc:194] Re-registered known tserver with Master: 933fc011f83a4f16a77ddea9e4d2936d (127.30.54.66:45907)
I20260808 07:10:39.193219 7137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:52483
W20260808 07:10:39.271209 8727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:39.271409 8727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:39.271432 8727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:39.272567 8727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:39.272606 8727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:39.272632 8727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:10:39.274003 8727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab
--principal=oryx/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:42685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=35913
--tserver_master_addrs=127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:10:39.274174 8727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:39.274339 8727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:10:39.279460 8727 server_base.cc:1034] running on GCE node
W20260808 07:10:39.279415 8732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:39.279428 8734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:39.279433 8737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:39.279954 8727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:39.280198 8727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:39.281343 8727 hybrid_clock.cc:648] HybridClock initialized: now 1786173039281332 us; error 29 us; skew 500 ppm
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173039, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:39.284026 8727 init.cc:377] Logged in from keytab as oryx/127.30.54.67@KRBTEST.COM (short username oryx)
I20260808 07:10:39.284508 8727 webserver.cc:460] Webserver started at http://127.30.54.67:35913/ using document root <none> and password file <none>
I20260808 07:10:39.284739 8727 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:39.284829 8727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:39.286312 8727 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:39.287042 8746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:39.287238 8727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:39.287352 8727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "cd43768cb6c141a6a81a65237b3d7420"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-znh6"
server_key: "a668c81a92a060e7b4c50d61ebff7431"
server_key_iv: "228df9a1cc0bc11fab362e998b31c38a"
server_key_version: "encryptionkey@0"
I20260808 07:10:39.287679 8727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:39.308476 8727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:39.310357 8727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:39.310520 8727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:39.310750 8727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:39.311075 8727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:39.311133 8727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:39.311173 8727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:39.311200 8727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:39.317322 8727 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:42685
I20260808 07:10:39.317375 8922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:42685 every 8 connection(s)
I20260808 07:10:39.317800 8727 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:39.323155 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 8727
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173039, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173039, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.124@KRBTEST.COM
Aug 08 07:10:39 dist-test-slave-znh6 krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173039, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.125@KRBTEST.COM
I20260808 07:10:39.330025 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.318944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:49651 (local address 127.30.54.124:42045)
0808 07:10:39.319019 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.319021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.319114 (+ 93us) server_negotiation.cc:407] Connection header received
0808 07:10:39.319605 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.319606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.319616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.319679 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.320534 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.321265 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.322151 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.322438 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.322505 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.326695 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.326697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.326697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.326704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.328735 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.329150 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.329151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.329151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.329207 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.329416 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.329417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.329417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.329545 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.329603 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.329851 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.329934 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:39.330164 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.318524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:43597)
0808 07:10:39.318693 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.318727 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.319506 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.319741 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.319745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.320246 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:39.320395 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.320401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.321143 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.321144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.322238 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.322239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.322722 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.322723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.322802 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.323275 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:39.323280 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:39.324994 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:39.327540 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.327548 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.327550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.328322 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.329034 (+ 712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.329035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.329036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.329169 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.329469 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:39.329476 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:39.329655 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.329764 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.329939 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":50,"thread_start_us":99,"threads_started":1}
I20260808 07:10:39.330341 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.318844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:42045 (local address 127.30.54.67:49651)
0808 07:10:39.318959 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.319042 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.319395 (+ 353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.319741 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.319745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.320180 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:39.320394 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.320398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.321380 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.321381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.322057 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.322059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.322525 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.322526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.322655 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.323135 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:39.323147 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:39.326251 (+ 3104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:39.328824 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.328825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.328826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.329072 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.329292 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.329293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.329293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.329345 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.329628 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:39.329629 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:39.329659 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.329764 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.330228 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":332,"thread_start_us":64,"threads_started":1}
I20260808 07:10:39.330679 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.318515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:43597 (local address 127.30.54.126:39687)
0808 07:10:39.318567 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.318569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.318835 (+ 266us) server_negotiation.cc:407] Connection header received
0808 07:10:39.319591 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.319593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.319605 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.319667 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.320514 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.321014 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.322341 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.322605 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.322646 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.325114 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.325116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.325116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.325124 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.327452 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.328649 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.328650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.328651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.328954 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.329244 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.329245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.329246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.329381 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.329444 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.330421 (+ 977us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.330550 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:10:39.330890 8923 heartbeater.cc:347] Connected to a master server at 127.30.54.124:42045
I20260808 07:10:39.330991 8923 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:39.331175 8923 heartbeater.cc:511] Master 127.30.54.124:42045 requested a full tablet report, sending...
I20260808 07:10:39.331197 8924 heartbeater.cc:347] Connected to a master server at 127.30.54.126:39687
I20260808 07:10:39.331245 8924 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:39.331329 8924 heartbeater.cc:511] Master 127.30.54.126:39687 requested a full tablet report, sending...
I20260808 07:10:39.331643 7137 ts_manager.cc:194] Re-registered known tserver with Master: cd43768cb6c141a6a81a65237b3d7420 (127.30.54.67:42685)
I20260808 07:10:39.331686 8235 ts_manager.cc:194] Registered new tserver with Master: cd43768cb6c141a6a81a65237b3d7420 (127.30.54.67:42685)
I20260808 07:10:39.332177 7137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:43597
I20260808 07:10:39.332791 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.318538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:39515 (local address 127.30.54.67:36385)
0808 07:10:39.318879 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.318888 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.319381 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.319740 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.319745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.320239 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:39.320395 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.320401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.321347 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.321347 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.322231 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.322233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.322682 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.322683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.322756 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.323889 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:39.323894 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:39.326584 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:39.331658 (+ 5074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.331659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.331659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.331884 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.332078 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:39.332079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:39.332079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:39.332133 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:39.332427 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:39.332428 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:39.332468 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.332559 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.332661 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":11,"spinlock_wait_cycles":9984,"thread_start_us":101,"threads_started":1}
I20260808 07:10:39.333112 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.318770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:36385 (local address 127.30.54.125:39515)
0808 07:10:39.318842 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.318844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.318935 (+ 91us) server_negotiation.cc:407] Connection header received
0808 07:10:39.319477 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.319478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.319493 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.319547 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.320498 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.321265 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.322317 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.322599 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.322680 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.326990 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.326991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.326991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.326999 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.331576 (+ 4577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.331963 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.331964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.331965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.332025 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.332216 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.332217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.332217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.332346 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.332434 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.332599 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.332729 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:10:39.333494 8925 heartbeater.cc:347] Connected to a master server at 127.30.54.125:39515
I20260808 07:10:39.333544 8925 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:39.333616 8925 heartbeater.cc:511] Master 127.30.54.125:39515 requested a full tablet report, sending...
I20260808 07:10:39.334090 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.324407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44030 (local address 127.30.54.125:39515)
0808 07:10:39.324544 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.324547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.325034 (+ 487us) server_negotiation.cc:407] Connection header received
0808 07:10:39.325059 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.325060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.325073 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.325149 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.325977 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.326733 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.327797 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.328102 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.328154 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.329703 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.329704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.329705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.329712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.332765 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.333182 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.333183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.333184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.333248 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.333492 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.333492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.333493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.333610 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.333712 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.333908 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.334012 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260808 07:10:39.334527 7979 ts_manager.cc:194] Registered new tserver with Master: cd43768cb6c141a6a81a65237b3d7420 (127.30.54.67:42685)
I20260808 07:10:39.335438 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.324535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43746 (local address 127.30.54.126:39687)
0808 07:10:39.324683 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.324685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.324989 (+ 304us) server_negotiation.cc:407] Connection header received
0808 07:10:39.325088 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.325088 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.325111 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.325211 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.326385 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.327113 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.328076 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.328369 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.328411 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.330399 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.330400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.330400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.330409 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.334344 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.334686 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.334688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.334688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.334749 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.334977 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.334978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.334979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.335102 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.335207 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.335263 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.335345 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260808 07:10:39.336921 7137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:39.338740 7137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:39.340636 8946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.325885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56732 (local address 127.30.54.124:42045)
0808 07:10:39.328742 (+ 2857us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.328744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.328753 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:10:39.328775 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.328776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.328824 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.328889 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.333488 (+ 4599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.335399 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.336344 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.336655 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.336723 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.337866 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.337868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.337868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.337875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.339445 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.339806 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.339807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.339808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.339864 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.340081 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.340082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.340082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.340202 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.340317 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.340449 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.340525 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2834,"thread_start_us":57,"threads_started":1}
I20260808 07:10:39.346266 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.342804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56570 (local address 127.30.54.67:42685)
0808 07:10:39.342995 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.342999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.343009 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:39.343046 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.343048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.343062 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.343162 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:39.343285 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.344060 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.345227 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.345933 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.345999 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.346072 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.346160 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.346195 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260808 07:10:39.346267 8951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.342599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:42685 (local address 127.0.0.1:56570)
0808 07:10:39.342724 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.342734 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.342776 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.343175 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.343176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.343178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:39.343221 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.343222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.344168 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.344168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.345140 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.345142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.346016 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.346017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.346077 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.346081 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.346148 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.346186 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":86,"threads_started":1}
I20260808 07:10:39.346760 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.342540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:38663 (local address 127.0.0.1:60238)
0808 07:10:39.342574 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.342585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.342623 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.343468 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.343468 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.343469 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:39.343505 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.343506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.344681 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.344682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.345572 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.345574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.346528 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.346529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.346590 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.346593 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.346660 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.346700 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 07:10:39.347061 8956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.342644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:45907 (local address 127.0.0.1:37426)
0808 07:10:39.343001 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.343010 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.343152 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.343364 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.343365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.343366 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:39.343395 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.343397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.344615 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.344616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.345753 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.345756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.346795 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.346796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.346898 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.346902 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.346955 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.346993 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260808 07:10:39.346784 8953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.342628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.30.54.65:38663)
0808 07:10:39.343302 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.343307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.343316 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:10:39.343352 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.343354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.343369 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.343471 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:39.343557 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.344592 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.345669 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.346291 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.346526 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.346609 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.346674 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.346710 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":80,"threads_started":1}
I20260808 07:10:39.347491 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.342890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.30.54.66:45907)
0808 07:10:39.343118 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.343123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.343133 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:39.343224 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.343227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.343244 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.343374 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:39.343441 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.344519 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.345854 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.346622 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.346699 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.346758 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.347172 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.347210 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":91,"threads_started":1}
I20260808 07:10:39.347359 8826 tablet_service.cc:1467] Processing CreateTablet for tablet 98f53145bf9f44a2b590c014c069ac38 (DEFAULT_TABLE table=test-table [id=419de4a17ae2477e89793e2b3775a28c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:39.347779 8826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98f53145bf9f44a2b590c014c069ac38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:39.347882 8412 tablet_service.cc:1467] Processing CreateTablet for tablet 98f53145bf9f44a2b590c014c069ac38 (DEFAULT_TABLE table=test-table [id=419de4a17ae2477e89793e2b3775a28c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:39.348107 8623 tablet_service.cc:1467] Processing CreateTablet for tablet 98f53145bf9f44a2b590c014c069ac38 (DEFAULT_TABLE table=test-table [id=419de4a17ae2477e89793e2b3775a28c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:39.348273 8412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98f53145bf9f44a2b590c014c069ac38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:39.348418 8623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98f53145bf9f44a2b590c014c069ac38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:39.351814 8962 tablet_bootstrap.cc:492] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: Bootstrap starting.
I20260808 07:10:39.351895 8961 tablet_bootstrap.cc:492] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: Bootstrap starting.
I20260808 07:10:39.351996 8960 tablet_bootstrap.cc:492] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: Bootstrap starting.
I20260808 07:10:39.352509 8962 tablet_bootstrap.cc:654] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:39.352509 8961 tablet_bootstrap.cc:654] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:39.352531 8960 tablet_bootstrap.cc:654] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:39.352834 8960 log.cc:824] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:39.352846 8961 log.cc:824] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:39.352856 8962 log.cc:824] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:39.353606 8960 tablet_bootstrap.cc:492] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: No bootstrap required, opened a new log
I20260808 07:10:39.353701 8960 ts_tablet_manager.cc:1397] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:39.353885 8961 tablet_bootstrap.cc:492] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: No bootstrap required, opened a new log
I20260808 07:10:39.353956 8961 ts_tablet_manager.cc:1397] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:39.354172 8962 tablet_bootstrap.cc:492] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: No bootstrap required, opened a new log
I20260808 07:10:39.354247 8962 ts_tablet_manager.cc:1397] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:39.355464 8961 raft_consensus.cc:348] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } }
I20260808 07:10:39.355614 8961 raft_consensus.cc:374] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:39.355659 8961 raft_consensus.cc:729] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd43768cb6c141a6a81a65237b3d7420, State: Initialized, Role: FOLLOWER
I20260808 07:10:39.355769 8961 consensus_queue.cc:260] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } }
I20260808 07:10:39.356068 8924 heartbeater.cc:503] Master 127.30.54.126:39687 was elected leader, sending a full tablet report...
I20260808 07:10:39.356812 8962 raft_consensus.cc:348] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } }
I20260808 07:10:39.356963 8962 raft_consensus.cc:374] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:39.357003 8962 raft_consensus.cc:729] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933fc011f83a4f16a77ddea9e4d2936d, State: Initialized, Role: FOLLOWER
I20260808 07:10:39.356967 8960 raft_consensus.cc:348] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } }
I20260808 07:10:39.357108 8960 raft_consensus.cc:374] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:39.357125 8962 consensus_queue.cc:260] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } }
I20260808 07:10:39.357170 8961 ts_tablet_manager.cc:1428] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:39.357391 8962 ts_tablet_manager.cc:1428] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:39.357167 8960 raft_consensus.cc:729] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 762dda4727d845a5bb56adb9828d3308, State: Initialized, Role: FOLLOWER
I20260808 07:10:39.357548 8715 heartbeater.cc:503] Master 127.30.54.126:39687 was elected leader, sending a full tablet report...
I20260808 07:10:39.357575 8960 consensus_queue.cc:260] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } }
I20260808 07:10:39.358489 8960 ts_tablet_manager.cc:1428] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260808 07:10:39.358662 8517 heartbeater.cc:503] Master 127.30.54.126:39687 was elected leader, sending a full tablet report...
I20260808 07:10:39.366657 8970 raft_consensus.cc:482] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:39.366761 8970 raft_consensus.cc:504] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } }
I20260808 07:10:39.367166 8970 leader_election.cc:288] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 762dda4727d845a5bb56adb9828d3308 (127.30.54.65:38663), 933fc011f83a4f16a77ddea9e4d2936d (127.30.54.66:45907)
I20260808 07:10:39.370867 8959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.367525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:35935 (local address 127.30.54.66:45907)
0808 07:10:39.367577 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.367580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.367747 (+ 167us) server_negotiation.cc:407] Connection header received
0808 07:10:39.367812 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.367813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.367822 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.367884 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:39.367958 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.368613 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.369901 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.370461 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.370552 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.370561 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.370729 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.370766 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:10:39.370879 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.367340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:38663 (local address 127.30.54.67:37143)
0808 07:10:39.367394 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.367406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.367446 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.367613 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.367614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.367616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:39.367681 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.367682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.368588 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.368588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.369777 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.369779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.370565 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.370566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.370653 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.370656 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.370707 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.370755 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":4}
I20260808 07:10:39.370904 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.367645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:45907 (local address 127.30.54.67:35935)
0808 07:10:39.367684 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:39.367695 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:39.367736 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:39.367884 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:39.367884 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:39.367885 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:39.367923 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.367923 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.368680 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.368681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.369777 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:39.369779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.370565 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.370566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:39.370644 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.370648 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:39.370707 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:10:39.370755 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":17}
I20260808 07:10:39.370987 8953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.367390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:37143 (local address 127.30.54.65:38663)
0808 07:10:39.367433 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.367435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.367442 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:10:39.367522 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.367523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.367533 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.367598 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:39.367763 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.368511 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.369901 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.370490 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.370555 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.370562 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.370729 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.370766 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:10:39.371234 8439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98f53145bf9f44a2b590c014c069ac38" candidate_uuid: "cd43768cb6c141a6a81a65237b3d7420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "762dda4727d845a5bb56adb9828d3308" is_pre_election: true
I20260808 07:10:39.371255 8651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98f53145bf9f44a2b590c014c069ac38" candidate_uuid: "cd43768cb6c141a6a81a65237b3d7420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933fc011f83a4f16a77ddea9e4d2936d" is_pre_election: true
I20260808 07:10:39.371399 8651 raft_consensus.cc:2454] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd43768cb6c141a6a81a65237b3d7420 in term 0.
I20260808 07:10:39.371421 8439 raft_consensus.cc:2454] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd43768cb6c141a6a81a65237b3d7420 in term 0.
I20260808 07:10:39.371599 8751 leader_election.cc:302] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 933fc011f83a4f16a77ddea9e4d2936d, cd43768cb6c141a6a81a65237b3d7420; no voters:
I20260808 07:10:39.371743 8970 raft_consensus.cc:2781] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:39.371815 8970 raft_consensus.cc:482] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:39.371850 8970 raft_consensus.cc:3037] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:39.372479 8970 raft_consensus.cc:504] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } }
I20260808 07:10:39.372608 8970 leader_election.cc:288] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [CANDIDATE]: Term 1 election: Requested vote from peers 762dda4727d845a5bb56adb9828d3308 (127.30.54.65:38663), 933fc011f83a4f16a77ddea9e4d2936d (127.30.54.66:45907)
I20260808 07:10:39.372800 8439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98f53145bf9f44a2b590c014c069ac38" candidate_uuid: "cd43768cb6c141a6a81a65237b3d7420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "762dda4727d845a5bb56adb9828d3308"
I20260808 07:10:39.372802 8651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98f53145bf9f44a2b590c014c069ac38" candidate_uuid: "cd43768cb6c141a6a81a65237b3d7420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933fc011f83a4f16a77ddea9e4d2936d"
I20260808 07:10:39.372884 8651 raft_consensus.cc:3037] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:39.372887 8439 raft_consensus.cc:3037] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:39.373674 8651 raft_consensus.cc:2454] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd43768cb6c141a6a81a65237b3d7420 in term 1.
I20260808 07:10:39.373797 8439 raft_consensus.cc:2454] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd43768cb6c141a6a81a65237b3d7420 in term 1.
I20260808 07:10:39.373842 8751 leader_election.cc:302] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 933fc011f83a4f16a77ddea9e4d2936d, cd43768cb6c141a6a81a65237b3d7420; no voters:
I20260808 07:10:39.373914 8970 raft_consensus.cc:2781] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:39.374056 8970 raft_consensus.cc:686] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 1 LEADER]: Becoming Leader. State: Replica: cd43768cb6c141a6a81a65237b3d7420, State: Running, Role: LEADER
I20260808 07:10:39.374147 8970 consensus_queue.cc:237] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } }
I20260808 07:10:39.374989 7135 catalog_manager.cc:5495] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd43768cb6c141a6a81a65237b3d7420 (127.30.54.67). New cstate: current_term: 1 leader_uuid: "cd43768cb6c141a6a81a65237b3d7420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd43768cb6c141a6a81a65237b3d7420" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 42685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:39.385340 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.383181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.30.54.67:42685)
0808 07:10:39.383246 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.383249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.383268 (+ 19us) server_negotiation.cc:407] Connection header received
0808 07:10:39.383302 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.383302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.383319 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.383378 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:39.383493 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.383952 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.384654 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.384845 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.384903 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.385042 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:39.385129 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:39.385187 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.385229 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.385247 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:10:39.387233 8651 raft_consensus.cc:1261] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [term 1 FOLLOWER]: Refusing update from remote peer cd43768cb6c141a6a81a65237b3d7420: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:39.387233 8439 raft_consensus.cc:1261] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [term 1 FOLLOWER]: Refusing update from remote peer cd43768cb6c141a6a81a65237b3d7420: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:39.387588 8970 consensus_queue.cc:1035] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "762dda4727d845a5bb56adb9828d3308" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:39.387722 8973 consensus_queue.cc:1035] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "933fc011f83a4f16a77ddea9e4d2936d" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:39.390887 8980 mvcc.cc:205] Tried to move back new op lower bound from 7316164769327456256 to 7316164769277325312. Current Snapshot: MvccSnapshot[applied={T|T < 7316164769327456256}]
I20260808 07:10:39.392341 8979 mvcc.cc:205] Tried to move back new op lower bound from 7316164769327456256 to 7316164769277325312. Current Snapshot: MvccSnapshot[applied={T|T < 7316164769327456256}]
I20260808 07:10:39.392802 7134 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:39.392895 7134 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:39.394204 7134 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bec56ea19cb14faebd957b877001c29b: Sending DeleteTablet for 3 replicas of tablet 98f53145bf9f44a2b590c014c069ac38
I20260808 07:10:39.394573 8826 tablet_service.cc:1514] Processing DeleteTablet for tablet 98f53145bf9f44a2b590c014c069ac38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:39 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:56570
I20260808 07:10:39.394863 8623 tablet_service.cc:1514] Processing DeleteTablet for tablet 98f53145bf9f44a2b590c014c069ac38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:39 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:37426
I20260808 07:10:39.395256 8412 tablet_service.cc:1514] Processing DeleteTablet for tablet 98f53145bf9f44a2b590c014c069ac38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:39 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:60238
I20260808 07:10:39.395284 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 7094
I20260808 07:10:39.398917 8996 tablet_replica.cc:330] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: stopping tablet replica
I20260808 07:10:39.399101 8996 raft_consensus.cc:2229] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:39.399317 8996 raft_consensus.cc:2258] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:39.404157 8995 tablet_replica.cc:330] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: stopping tablet replica
I20260808 07:10:39.404273 8995 raft_consensus.cc:2229] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:39.404384 8995 raft_consensus.cc:2258] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:39.404881 8995 ts_tablet_manager.cc:1905] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:39.406620 8993 tablet_replica.cc:330] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: stopping tablet replica
I20260808 07:10:39.406731 8993 raft_consensus.cc:2229] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:39.406899 8993 raft_consensus.cc:2258] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:39.407085 8995 ts_tablet_manager.cc:1918] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:39.407164 8995 log.cc:1196] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/98f53145bf9f44a2b590c014c069ac38
I20260808 07:10:39.407467 8995 ts_tablet_manager.cc:1939] T 98f53145bf9f44a2b590c014c069ac38 P 933fc011f83a4f16a77ddea9e4d2936d: Deleting consensus metadata
I20260808 07:10:39.407459 8996 ts_tablet_manager.cc:1905] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:39.407483 8993 ts_tablet_manager.cc:1905] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:39.408980 8996 ts_tablet_manager.cc:1918] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:39.409060 8996 log.cc:1196] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/98f53145bf9f44a2b590c014c069ac38
I20260808 07:10:39.409343 8996 ts_tablet_manager.cc:1939] T 98f53145bf9f44a2b590c014c069ac38 P 762dda4727d845a5bb56adb9828d3308: Deleting consensus metadata
I20260808 07:10:39.409735 8993 ts_tablet_manager.cc:1918] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:39.409853 8993 log.cc:1196] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/98f53145bf9f44a2b590c014c069ac38
I20260808 07:10:39.410156 8993 ts_tablet_manager.cc:1939] T 98f53145bf9f44a2b590c014c069ac38 P cd43768cb6c141a6a81a65237b3d7420: Deleting consensus metadata
W20260808 07:10:39.412163 8556 connection.cc:535] server connection from 127.0.0.1:37426 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:10:39.412169 8751 connection.cc:535] server connection from 127.0.0.1:56570 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:10:39.412204 8334 connection.cc:535] server connection from 127.0.0.1:60238 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:10:39.422281 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.414094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.30.54.124:42045)
0808 07:10:39.414135 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.414137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.414229 (+ 92us) server_negotiation.cc:407] Connection header received
0808 07:10:39.414340 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.414341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.414353 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.414403 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.415256 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.416081 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.417155 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.417469 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.417537 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.419265 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.419266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.419266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.419274 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.420888 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.421293 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.421294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.421294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.421351 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.421555 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.421556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.421556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.421809 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.421891 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.422106 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.422185 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:39.422592 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.413963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44038 (local address 127.30.54.125:39515)
0808 07:10:39.414008 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.414010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.414205 (+ 195us) server_negotiation.cc:407] Connection header received
0808 07:10:39.414261 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.414262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.414279 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.414342 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.415506 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.415999 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.417175 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.417380 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.417468 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.419085 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.419087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.419088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.419112 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.421266 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.421658 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.421659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.421659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.421728 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.421916 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.421917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.421917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.422050 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.422120 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.422351 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.422469 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:10:39.426132 8946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:39.395092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.30.54.124:42045)
0808 07:10:39.395135 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:39.395137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:39.409302 (+ 14165us) server_negotiation.cc:407] Connection header received
0808 07:10:39.419021 (+ 9719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:39.419023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:39.419040 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:39.419072 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:39.419757 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.420544 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.421614 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:39.421976 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:39.422019 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:39.423498 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:39.423500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:39.423500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:39.423507 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:39.425039 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.425355 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.425356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.425357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.425418 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:39.425599 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:39.425600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:39.425600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:39.425729 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:39.425838 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:39.425952 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 07:10:39.426038 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:40.392513 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:40.392198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:33235)
0808 07:10:40.392355 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:40.392450 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
W20260808 07:10:40.392747 8517 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:39687 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:40.392573 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:40.392250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:33537)
0808 07:10:40.392433 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:40.392525 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":83,"threads_started":1}
W20260808 07:10:40.393121 8715 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:39687 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:40.393179 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:40.392936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:46555)
0808 07:10:40.393064 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:40.393135 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
W20260808 07:10:40.393345 8924 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:39687 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:40.894564 9010 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bec56ea19cb14faebd957b877001c29b)
I20260808 07:10:40.894698 9010 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:10:40.895005 9010 leader_election.cc:288] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687)
I20260808 07:10:40.895421 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:40.895195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:44244)
0808 07:10:40.895318 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:40.895380 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
W20260808 07:10:40.895606 7962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:40.896034 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:40.895921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:44260)
0808 07:10:40.895979 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:40.896010 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:10:40.896140 7962 leader_election.cc:334] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687): Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:40.896229 7962 leader_election.cc:302] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ce4b9e938cd94ec1b42843f52bd9936e; no voters: bec56ea19cb14faebd957b877001c29b
I20260808 07:10:40.896364 9010 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:41.393365 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:41.393112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:53659)
0808 07:10:41.393262 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:41.393303 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260808 07:10:41.393836 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:41.393458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:45279)
0808 07:10:41.393732 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:41.393770 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":57,"threads_started":1}
I20260808 07:10:41.393992 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:41.393693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:51661)
0808 07:10:41.393869 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:41.393947 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":79,"threads_started":1}
I20260808 07:10:42.394186 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:42.393909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:50673)
0808 07:10:42.394073 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:42.394125 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260808 07:10:42.394476 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:42.394199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:40693)
0808 07:10:42.394389 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:42.394427 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":53,"threads_started":1}
I20260808 07:10:42.394768 9019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:42.394509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:48839)
0808 07:10:42.394674 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:42.394716 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":56,"threads_started":1}
I20260808 07:10:42.453825 9020 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bec56ea19cb14faebd957b877001c29b)
I20260808 07:10:42.453953 9020 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:10:42.454152 9020 leader_election.cc:288] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687)
I20260808 07:10:42.454501 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:42.454300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:44514)
0808 07:10:42.454407 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:42.454451 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":46,"threads_started":1}
I20260808 07:10:42.454959 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:42.454859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:44530)
0808 07:10:42.454904 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:42.454939 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:10:42.455076 7962 leader_election.cc:334] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687): Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:42.455142 7962 leader_election.cc:302] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ce4b9e938cd94ec1b42843f52bd9936e; no voters: bec56ea19cb14faebd957b877001c29b
I20260808 07:10:42.455214 9020 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:43.395072 9024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:43.394718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:38679)
0808 07:10:43.394962 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:43.395011 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":193,"threads_started":1}
I20260808 07:10:43.395329 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:43.395138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:56311)
0808 07:10:43.395247 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:43.395287 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260808 07:10:43.395406 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:43.394957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:37343)
0808 07:10:43.395322 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:43.395366 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":60,"threads_started":1}
I20260808 07:10:44.396152 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:44.395907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:58651)
0808 07:10:44.396085 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:44.396115 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":71,"threads_started":1}
I20260808 07:10:44.396152 9027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:44.395889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:50451)
0808 07:10:44.396052 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:44.396097 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
I20260808 07:10:44.396353 9028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:44.395922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:47195)
0808 07:10:44.396248 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:44.396290 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":59,"threads_started":1}
I20260808 07:10:45.396879 9032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:45.396601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:33133)
0808 07:10:45.396774 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:45.396819 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":69,"threads_started":1}
I20260808 07:10:45.397048 9033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:45.396752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:58781)
0808 07:10:45.396938 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:45.396977 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":73,"threads_started":1}
I20260808 07:10:45.397172 9034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:45.396947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:52007)
0808 07:10:45.397099 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:45.397129 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":50,"threads_started":1}
I20260808 07:10:45.804493 9037 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bec56ea19cb14faebd957b877001c29b)
I20260808 07:10:45.804596 9037 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:10:45.804800 9037 leader_election.cc:288] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687)
I20260808 07:10:45.805158 9039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:45.804956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:57336)
0808 07:10:45.805074 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:45.805118 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":50,"threads_started":1}
I20260808 07:10:45.805622 9039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:45.805519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:57352)
0808 07:10:45.805567 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:45.805596 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:10:45.805709 7962 leader_election.cc:334] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687): Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:45.805799 7962 leader_election.cc:302] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ce4b9e938cd94ec1b42843f52bd9936e; no voters: bec56ea19cb14faebd957b877001c29b
I20260808 07:10:45.805877 9037 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:46.397760 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:46.397525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:44671)
0808 07:10:46.397657 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:46.397701 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260808 07:10:46.397790 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:46.397553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:45573)
0808 07:10:46.397715 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:46.397746 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":62,"threads_started":1}
I20260808 07:10:46.397790 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:46.397553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:50011)
0808 07:10:46.397710 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:46.397746 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":58,"threads_started":1}
I20260808 07:10:47.398533 9049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:47.398252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:55415)
0808 07:10:47.398402 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:47.398443 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260808 07:10:47.398979 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:47.398475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:59305)
0808 07:10:47.398876 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:47.398919 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":70,"threads_started":1}
I20260808 07:10:47.399170 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:47.398785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:41209)
0808 07:10:47.399068 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:47.399109 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":95,"threads_started":1}
I20260808 07:10:48.399859 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:48.399573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:37569)
0808 07:10:48.399708 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:48.399778 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260808 07:10:48.400336 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:48.399662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:40427)
0808 07:10:48.400232 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:48.400276 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":54,"threads_started":1}
I20260808 07:10:48.400579 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:48.399451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:44853)
0808 07:10:48.400499 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:48.400537 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1028,"thread_start_us":67,"threads_started":1}
I20260808 07:10:49.400628 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:49.400367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:52535)
0808 07:10:49.400505 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:49.400571 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260808 07:10:49.400928 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:49.400725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:47315)
0808 07:10:49.400837 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:49.400878 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
I20260808 07:10:49.401151 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:49.400950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:44427)
0808 07:10:49.401073 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:49.401105 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260808 07:10:50.110926 9061 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bec56ea19cb14faebd957b877001c29b)
I20260808 07:10:50.111027 9061 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:10:50.111181 9061 leader_election.cc:288] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687)
I20260808 07:10:50.111533 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:50.111349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:57684)
0808 07:10:50.111451 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:50.111491 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":48,"threads_started":1}
W20260808 07:10:50.111650 7962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260808 07:10:50.112083 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:50.111985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:57690)
0808 07:10:50.112033 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:50.112061 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:10:50.112165 7962 leader_election.cc:334] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687): Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:50.112236 7962 leader_election.cc:302] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ce4b9e938cd94ec1b42843f52bd9936e; no voters: bec56ea19cb14faebd957b877001c29b
I20260808 07:10:50.112320 9061 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:50.401458 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:50.401114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:55959)
0808 07:10:50.401272 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:50.401324 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260808 07:10:50.401702 9067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:50.401516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:53497)
0808 07:10:50.401630 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:50.401660 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260808 07:10:50.401641 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:50.401290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:47705)
0808 07:10:50.401550 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:50.401589 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":64,"threads_started":1}
I20260808 07:10:51.402275 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:51.402045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:50071)
0808 07:10:51.402153 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:51.402225 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":65,"threads_started":1}
I20260808 07:10:51.402271 9069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:51.402010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:42197)
0808 07:10:51.402160 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:51.402205 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260808 07:10:51.402541 9077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:51.402288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:52055)
0808 07:10:51.402464 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:51.402506 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":54,"threads_started":1}
I20260808 07:10:52.402980 9080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:52.402699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:38355)
0808 07:10:52.402870 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:52.402917 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
I20260808 07:10:52.403030 9079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:52.402709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:49041)
0808 07:10:52.402896 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:52.402952 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":70,"threads_started":1}
I20260808 07:10:52.403240 9082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:52.402980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:43239)
0808 07:10:52.403156 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:52.403191 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260808 07:10:53.403751 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:53.403452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:39237)
0808 07:10:53.403635 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:53.403683 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260808 07:10:53.403844 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:53.403609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:50525)
0808 07:10:53.403724 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:53.403766 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260808 07:10:53.403924 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:53.403692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:56991)
0808 07:10:53.403841 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:53.403879 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260808 07:10:54.404764 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:54.404313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:57593)
0808 07:10:54.404693 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:54.404722 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":67,"threads_started":1}
I20260808 07:10:54.404764 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:54.404493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:55465)
0808 07:10:54.404674 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:54.404715 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":75,"threads_started":1}
I20260808 07:10:54.404960 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:54.404525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:44541)
0808 07:10:54.404889 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:54.404923 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":54,"threads_started":1}
I20260808 07:10:55.405545 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:55.405318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:52843)
0808 07:10:55.405453 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:55.405491 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260808 07:10:55.405661 9094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:55.405345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:54253)
0808 07:10:55.405589 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:55.405622 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":61,"threads_started":1}
I20260808 07:10:55.405661 9095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:55.405453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:38239)
0808 07:10:55.405573 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:55.405614 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260808 07:10:56.406420 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:56.406159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:51109)
0808 07:10:56.406301 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:56.406344 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260808 07:10:56.406461 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:56.406096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:39635)
0808 07:10:56.406342 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:56.406385 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":70,"threads_started":1}
I20260808 07:10:56.406531 9100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:56.406165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:41953)
0808 07:10:56.406438 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:56.406485 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":45,"threads_started":1}
I20260808 07:10:56.554177 9103 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bec56ea19cb14faebd957b877001c29b)
I20260808 07:10:56.554270 9103 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:10:56.554500 9103 leader_election.cc:288] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687)
I20260808 07:10:56.554948 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:56.554716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:44228)
0808 07:10:56.554863 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:56.554907 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
W20260808 07:10:56.555037 7962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:56.555387 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:56.555291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:44234)
0808 07:10:56.555342 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:56.555369 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:10:56.555493 7962 leader_election.cc:334] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687): Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:10:56.555552 7962 leader_election.cc:302] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ce4b9e938cd94ec1b42843f52bd9936e; no voters: bec56ea19cb14faebd957b877001c29b
I20260808 07:10:56.555629 9103 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:57.407136 9106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:57.406896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:41169)
0808 07:10:57.407032 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:57.407077 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260808 07:10:57.407214 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:57.406967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:36509)
0808 07:10:57.407121 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:57.407164 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":60,"threads_started":1}
I20260808 07:10:57.407764 9109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:57.407520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:56707)
0808 07:10:57.407692 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:57.407723 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":57,"threads_started":1}
I20260808 07:10:58.407894 9111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:58.407613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:45013)
0808 07:10:58.407789 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:58.407835 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":50,"threads_started":1}
I20260808 07:10:58.407922 9110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:58.407596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:41541)
0808 07:10:58.407848 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:58.407880 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":68,"threads_started":1}
I20260808 07:10:58.408563 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:58.408251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:37851)
0808 07:10:58.408476 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:58.408514 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":52,"threads_started":1}
I20260808 07:10:59.408816 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:59.408473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:53927)
0808 07:10:59.408652 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:59.408701 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260808 07:10:59.408881 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:59.408447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:55979)
0808 07:10:59.408661 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:59.408706 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260808 07:10:59.409179 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:59.408960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:50961)
0808 07:10:59.409093 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:59.409132 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260808 07:11:00.409584 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.409362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:38511)
0808 07:11:00.409479 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.409521 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
I20260808 07:11:00.409999 9120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.409637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:48979)
0808 07:11:00.409913 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.409952 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":70,"threads_started":1}
I20260808 07:11:00.410012 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.409325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:52695)
0808 07:11:00.409913 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.409955 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":108,"threads_started":1}
I20260808 07:11:01.410440 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:01.410156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:54797)
0808 07:11:01.410339 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:01.410380 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
I20260808 07:11:01.410617 9123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:01.410403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:46353)
0808 07:11:01.410524 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:01.410562 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":93,"threads_started":1}
I20260808 07:11:01.410884 9124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:01.410445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:33763)
0808 07:11:01.410750 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:01.410793 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":57,"threads_started":1}
I20260808 07:11:02.411206 9125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:02.410962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:51963)
0808 07:11:02.411106 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:02.411149 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260808 07:11:02.411516 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:02.411255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:34519)
0808 07:11:02.411413 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:02.411473 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":57,"threads_started":1}
I20260808 07:11:02.411619 9128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:02.411318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:40547)
0808 07:11:02.411538 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:02.411571 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":74,"threads_started":1}
I20260808 07:11:02.702183 9129 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bec56ea19cb14faebd957b877001c29b)
I20260808 07:11:02.702294 9129 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bec56ea19cb14faebd957b877001c29b" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39687 } } peers { permanent_uuid: "ce4b9e938cd94ec1b42843f52bd9936e" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 39515 } attrs { promote: false } } peers { permanent_uuid: "8646d7de064b4751b36bea7e3f35e304" member_type: NON_VOTER last_known_addr { host: "127.30.54.124" port: 42045 } attrs { promote: true } }
I20260808 07:11:02.702466 9129 leader_election.cc:288] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687)
I20260808 07:11:02.702862 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:02.702621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:44514)
0808 07:11:02.702744 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:02.702787 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
W20260808 07:11:02.702957 7962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:11:02.703337 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:02.703236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.0.0.1:44526)
0808 07:11:02.703294 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:02.703320 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 07:11:02.703418 7962 leader_election.cc:334] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bec56ea19cb14faebd957b877001c29b (127.30.54.126:39687): Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
I20260808 07:11:02.703469 7962 leader_election.cc:302] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ce4b9e938cd94ec1b42843f52bd9936e; no voters: bec56ea19cb14faebd957b877001c29b
I20260808 07:11:02.703576 9129 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ce4b9e938cd94ec1b42843f52bd9936e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:11:03.412079 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.411862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:55959)
0808 07:11:03.411994 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.412029 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260808 07:11:03.412079 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.411864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:55001)
0808 07:11:03.411992 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.412030 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260808 07:11:03.412402 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.412113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:34689)
0808 07:11:03.412324 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.412366 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":70,"threads_started":1}
I20260808 07:11:04.412669 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:04.412425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:37539)
0808 07:11:04.412563 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:04.412607 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260808 07:11:04.412870 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:04.412577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:53931)
0808 07:11:04.412781 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:04.412817 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":56,"threads_started":1}
I20260808 07:11:04.412995 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:04.412729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:40733)
0808 07:11:04.412914 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:04.412952 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":53,"threads_started":1}
I20260808 07:11:05.413466 9139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:05.413209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:32783)
0808 07:11:05.413359 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:05.413406 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260808 07:11:05.413579 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:05.413345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:55181)
0808 07:11:05.413469 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:05.413516 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260808 07:11:05.413801 9143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:05.413559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:45089)
0808 07:11:05.413711 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:05.413762 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":50,"threads_started":1}
I20260808 07:11:06.414227 9145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:06.413916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:46687)
0808 07:11:06.414125 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:06.414168 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":71,"threads_started":1}
I20260808 07:11:06.414423 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:06.414206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:48627)
0808 07:11:06.414321 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:06.414367 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
I20260808 07:11:06.414662 9153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:06.414433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:46065)
0808 07:11:06.414574 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:06.414614 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260808 07:11:07.414999 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:07.414660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:48911)
0808 07:11:07.414893 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:07.414939 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260808 07:11:07.415091 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:07.414873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:43495)
0808 07:11:07.415000 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:07.415039 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260808 07:11:07.415230 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:07.414967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:47977)
0808 07:11:07.415083 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:07.415121 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260808 07:11:08.415732 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:08.415475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:40525)
0808 07:11:08.415628 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:08.415673 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260808 07:11:08.415753 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:08.415495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:54981)
0808 07:11:08.415675 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:08.415710 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260808 07:11:08.415753 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:08.415569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:41131)
0808 07:11:08.415673 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:08.415708 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260808 07:11:09.416501 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:09.416213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.65:38865)
0808 07:11:09.416411 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:09.416449 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":55,"threads_started":1}
I20260808 07:11:09.416536 9165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:09.416204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.66:55969)
0808 07:11:09.416340 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:09.416384 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260808 07:11:09.416585 9164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:09.416204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39687 (local address 127.30.54.67:56673)
0808 07:11:09.416344 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:09.416389 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:39687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.30.54.126:39687,127.30.54.125:39515,127.30.54.124:42045, num_attempts: 1)
I20260808 07:11:09.612586 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 8187
I20260808 07:11:09.628571 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 8530
I20260808 07:11:09.634333 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 8727
I20260808 07:11:09.643218 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 7796
I20260808 07:11:09.651309 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 8057
2026-08-08T07:11:09Z chronyd exiting
I20260808 07:11:09.670149 30937 test_util.cc:175] -----------------------------------------------
I20260808 07:11:09.670204 30937 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786173009519897-30937-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (33394 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:09 dist-test-slave-znh6 krb5kdc[9170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:09 dist-test-slave-znh6 krb5kdc[9170](info): set up 2 sockets
Aug 08 07:11:09 dist-test-slave-znh6 krb5kdc[9170](info): commencing operation
krb5kdc: starting...
W20260808 07:11:10.878960 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:11:10 dist-test-slave-znh6 krb5kdc[9170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.126.keytab.
Entry for principal oryx/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.126.keytab.
I20260808 07:11:11.002372 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45043
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.126.keytab
--principal=oryx/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:32887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:45043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:11.084676 9190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:11.084863 9190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:11.084885 9190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:11.085935 9190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:11.085969 9190 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:11:11.085983 9190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:11.085996 9190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:11.086009 9190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:11.086020 9190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:11.087543 9190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:32887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:45043
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.126.keytab
--principal=oryx/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:11.087728 9190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:11.087896 9190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:11.092890 9203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:11.092864 9197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:11.092908 9190 server_base.cc:1034] running on GCE node
W20260808 07:11:11.092864 9207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:11.093350 9190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:11.093590 9190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:11.094741 9190 hybrid_clock.cc:648] HybridClock initialized: now 1786173071094729 us; error 32 us; skew 500 ppm
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:11.097231 9190 init.cc:377] Logged in from keytab as oryx/127.30.54.126@KRBTEST.COM (short username oryx)
I20260808 07:11:11.097654 9190 webserver.cc:460] Webserver started at http://127.30.54.126:35145/ using document root <none> and password file <none>
I20260808 07:11:11.097892 9190 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:11.097972 9190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:11.098124 9190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:11.099241 9190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "0b774c867ab84a4a8dec70240f335105"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "3c668b5b021aad26592015fd59af30d9"
server_key_iv: "b27c0b4b7d32c73cba51e414883337ca"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.099591 9190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "0b774c867ab84a4a8dec70240f335105"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "3c668b5b021aad26592015fd59af30d9"
server_key_iv: "b27c0b4b7d32c73cba51e414883337ca"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.100879 9190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:11.101586 9215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.101748 9190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:11.101853 9190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "0b774c867ab84a4a8dec70240f335105"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "3c668b5b021aad26592015fd59af30d9"
server_key_iv: "b27c0b4b7d32c73cba51e414883337ca"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.101927 9190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:11.116338 9190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:11.118019 9190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:11.118180 9190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:11.122900 9190 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:45043
I20260808 07:11:11.122905 9315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:45043 every 8 connection(s)
I20260808 07:11:11.123281 9190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:11.123906 9317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:11.126094 9317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105: Bootstrap starting.
I20260808 07:11:11.126688 9317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:11.127030 9317 log.cc:824] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:11.127645 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 9190
I20260808 07:11:11.127734 9317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105: No bootstrap required, opened a new log
I20260808 07:11:11.127732 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:11:11.127967 30937 external_mini_cluster.cc:1428] Setting key 164ca1712830870c730a3fd773851af3
I20260808 07:11:11.130636 9317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b774c867ab84a4a8dec70240f335105" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45043 } }
I20260808 07:11:11.130798 9317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:11.130900 9317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b774c867ab84a4a8dec70240f335105, State: Initialized, Role: FOLLOWER
I20260808 07:11:11.131022 9317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b774c867ab84a4a8dec70240f335105" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45043 } }
I20260808 07:11:11.131090 9317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:11.131116 9317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:11.131151 9317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:11.131778 9317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b774c867ab84a4a8dec70240f335105" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45043 } }
I20260808 07:11:11.131865 9317 leader_election.cc:302] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b774c867ab84a4a8dec70240f335105; no voters:
I20260808 07:11:11.132055 9317 leader_election.cc:288] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:11.132082 9324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:11.132192 9324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [term 1 LEADER]: Becoming Leader. State: Replica: 0b774c867ab84a4a8dec70240f335105, State: Running, Role: LEADER
I20260808 07:11:11.132323 9317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:11.132345 9324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b774c867ab84a4a8dec70240f335105" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45043 } }
I20260808 07:11:11.132699 9324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b774c867ab84a4a8dec70240f335105. Latest consensus state: current_term: 1 leader_uuid: "0b774c867ab84a4a8dec70240f335105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b774c867ab84a4a8dec70240f335105" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45043 } } }
I20260808 07:11:11.132712 9326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b774c867ab84a4a8dec70240f335105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b774c867ab84a4a8dec70240f335105" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45043 } } }
I20260808 07:11:11.132856 9324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:11.133083 9326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:11.133702 9336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:11.134243 9336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:11:11.135986 9336 catalog_manager.cc:1347] Generated new cluster ID: e9c64505077e4439bd04b719078525ab
I20260808 07:11:11.136041 9336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:11.139694 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.128497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37142 (local address 127.30.54.126:45043)
0808 07:11:11.128684 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.128692 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.128724 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:11:11.129216 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.129226 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.129285 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.129433 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:11.130025 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.131378 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.132430 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.133042 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.133124 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.136034 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:11.136038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:11.136038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:11.136050 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:11.137727 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.138107 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.138110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.138111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.138205 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.138425 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.138425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.138426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.138656 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:11.139381 (+ 725us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.139445 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.139551 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260808 07:11:11.142282 9336 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:11.142789 9336 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:11.155674 9336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105: Generated new TSK 0
I20260808 07:11:11.155879 9336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab.
Entry for principal oryx/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab.
I20260808 07:11:11.189539 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:45043
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab
--principal=oryx/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:32887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:11.290388 9356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:11.290578 9356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:11.290603 9356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:11.291788 9356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:11.291831 9356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:11.291851 9356 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:11.291910 9356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:11:11.293392 9356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:32887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab
--principal=oryx/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.54.126:45043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:11.293619 9356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:11.293862 9356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:11.299032 9367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:11.299037 9368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:11.299047 9370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:11.299464 9356 server_base.cc:1034] running on GCE node
I20260808 07:11:11.299686 9356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:11.299942 9356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:11.301111 9356 hybrid_clock.cc:648] HybridClock initialized: now 1786173071301096 us; error 30 us; skew 500 ppm
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:11.303660 9356 init.cc:377] Logged in from keytab as oryx/127.30.54.65@KRBTEST.COM (short username oryx)
I20260808 07:11:11.304075 9356 webserver.cc:460] Webserver started at http://127.30.54.65:43573/ using document root <none> and password file <none>
I20260808 07:11:11.304318 9356 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:11.304399 9356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:11.304597 9356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:11.305728 9356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "36c2ec8add924a13ae05aa23a2984a70"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "0e9182f30fd9571b321fa5049eb3aeae"
server_key_iv: "2a13179c858dd2d793c13e0f42d9610b"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.306082 9356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "36c2ec8add924a13ae05aa23a2984a70"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "0e9182f30fd9571b321fa5049eb3aeae"
server_key_iv: "2a13179c858dd2d793c13e0f42d9610b"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.307507 9356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:11.308247 9385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.308470 9356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:11.308573 9356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "36c2ec8add924a13ae05aa23a2984a70"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "0e9182f30fd9571b321fa5049eb3aeae"
server_key_iv: "2a13179c858dd2d793c13e0f42d9610b"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.308652 9356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:11.322706 9356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:11.324661 9356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:11.324833 9356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:11.325316 9356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:11.325389 9356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.325439 9356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:11.325464 9356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.332451 9356 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:43561
I20260808 07:11:11.332482 9544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:43561 every 8 connection(s)
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:11:11.332841 9356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:11.335006 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 9356
I20260808 07:11:11.335120 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:11.335366 30937 external_mini_cluster.cc:1428] Setting key 24bba8d925f37d3118358f2eb4998484
I20260808 07:11:11.335872 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.325902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:51985 (local address 127.30.54.126:45043)
0808 07:11:11.325958 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.325960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.326157 (+ 197us) server_negotiation.cc:407] Connection header received
0808 07:11:11.326639 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.326640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.326653 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.326730 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:11.327619 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.328485 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.330100 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.330457 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.330539 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.333180 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:11.333182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:11.333182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:11.333189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:11.334419 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.334773 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.334774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.334775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.334869 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.335155 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.335157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.335157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.335322 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:11.335400 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.335724 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.335810 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:11:11.336112 9397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.325901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45043 (local address 127.30.54.65:51985)
0808 07:11:11.326050 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.326082 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.326533 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.326863 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.326866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.327205 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:11.327500 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.327507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.328618 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.328619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.329994 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.329998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.330561 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.330562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.330656 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.331205 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:11.331215 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:11.333094 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:11.334495 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.334501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.334503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.334704 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.334959 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.334960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.334960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.335076 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.335399 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:11.335421 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:11.335575 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.335686 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.335868 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260808 07:11:11.336661 9546 heartbeater.cc:347] Connected to a master server at 127.30.54.126:45043
I20260808 07:11:11.336755 9546 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:11.336956 9546 heartbeater.cc:511] Master 127.30.54.126:45043 requested a full tablet report, sending...
I20260808 07:11:11.337447 9239 ts_manager.cc:194] Registered new tserver with Master: 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561)
I20260808 07:11:11.338053 9239 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:51985
WARNING: no policy specified for oryx/127.30.54.66@KRBTEST.COM; defaulting to no policy
I20260808 07:11:11.344388 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.338930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:42281 (local address 127.30.54.126:45043)
0808 07:11:11.338971 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.338973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.339106 (+ 133us) server_negotiation.cc:407] Connection header received
0808 07:11:11.339175 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.339176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.339185 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.339241 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:11.339548 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.340001 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.340758 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.340955 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.341035 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.341905 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:11.341907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:11.341908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:11.341914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:11.343270 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.343634 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.343635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.343635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.343694 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.343891 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.343892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.343892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.344002 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:11.344077 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.344228 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.344299 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:11.344403 9558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.338896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45043 (local address 127.30.54.65:42281)
0808 07:11:11.339031 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.339040 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.339119 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.339246 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.339247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.339422 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:11.339476 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.339477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.340091 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.340092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.340670 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.340672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.341038 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.341038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.341108 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.341538 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:11.341542 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:11.341821 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:11.343365 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.343366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.343366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.343551 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.343759 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.343760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.343760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.343826 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.344069 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:11.344070 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:11.344114 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.344201 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.344301 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab.
Entry for principal oryx/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab.
I20260808 07:11:11.362445 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:45043
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab
--principal=oryx/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:32887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:11.443835 9564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:11.444067 9564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:11.444105 9564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:11.445231 9564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:11.445299 9564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:11.445336 9564 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:11.445375 9564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:11:11.446956 9564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:32887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab
--principal=oryx/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.54.126:45043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:11.447211 9564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:11.447438 9564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:11.452440 9564 server_base.cc:1034] running on GCE node
W20260808 07:11:11.452404 9571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:11.452419 9574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:11.452400 9569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:11.452863 9564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:11.453133 9564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:11.454280 9564 hybrid_clock.cc:648] HybridClock initialized: now 1786173071454262 us; error 41 us; skew 500 ppm
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:11.456763 9564 init.cc:377] Logged in from keytab as oryx/127.30.54.66@KRBTEST.COM (short username oryx)
I20260808 07:11:11.457182 9564 webserver.cc:460] Webserver started at http://127.30.54.66:39501/ using document root <none> and password file <none>
I20260808 07:11:11.457397 9564 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:11.457441 9564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:11.457612 9564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:11.458599 9564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "eb44a37cb4864c66ac05348abba27b9e"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "627233451fe5ea1226d3cf2f8ed6882d"
server_key_iv: "c4e2cfec45e6b8c3be0737dafb9470a5"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.458969 9564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "eb44a37cb4864c66ac05348abba27b9e"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "627233451fe5ea1226d3cf2f8ed6882d"
server_key_iv: "c4e2cfec45e6b8c3be0737dafb9470a5"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.460291 9564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:11.461054 9582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.461243 9564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:11.461339 9564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "eb44a37cb4864c66ac05348abba27b9e"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "627233451fe5ea1226d3cf2f8ed6882d"
server_key_iv: "c4e2cfec45e6b8c3be0737dafb9470a5"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.461414 9564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:11.469420 9564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:11.471201 9564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:11.471346 9564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:11.471870 9564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:11.471935 9564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.471976 9564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:11.472002 9564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:11:11.479085 9564 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:37829
I20260808 07:11:11.479174 9749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:37829 every 8 connection(s)
I20260808 07:11:11.479455 9564 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:11.481778 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.472273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:41027 (local address 127.30.54.126:45043)
0808 07:11:11.472336 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.472338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.472529 (+ 191us) server_negotiation.cc:407] Connection header received
0808 07:11:11.473002 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.473003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.473017 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.473082 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:11.473928 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.474769 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.476030 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.476377 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.476469 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.479237 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:11.479239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:11.479240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:11.479248 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:11.480705 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.481073 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.481074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.481074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.481122 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.481306 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.481307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.481307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.481398 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:11.481445 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.481662 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.481718 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:11.481984 9594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.472268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45043 (local address 127.30.54.66:41027)
0808 07:11:11.472405 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.472442 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.472895 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.473153 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.473156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.473527 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:11.473804 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.473810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.474909 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.474909 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.475931 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.475934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.476489 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.476490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.476622 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.477146 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:11.477156 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:11.479137 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:11.480787 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.480793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.480794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.481006 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.481164 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.481164 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.481164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.481257 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.481456 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:11.481461 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:11.481560 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.481637 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.481744 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
I20260808 07:11:11.482472 9751 heartbeater.cc:347] Connected to a master server at 127.30.54.126:45043
I20260808 07:11:11.482551 9751 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:11.482750 9751 heartbeater.cc:511] Master 127.30.54.126:45043 requested a full tablet report, sending...
I20260808 07:11:11.483161 9238 ts_manager.cc:194] Registered new tserver with Master: eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829)
I20260808 07:11:11.483549 9238 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:41027
I20260808 07:11:11.487293 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 9564
I20260808 07:11:11.487432 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:11.487638 30937 external_mini_cluster.cc:1428] Setting key 4858196f35cfc0380cf9e505a4fca207
I20260808 07:11:11.489248 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.483522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45043 (local address 127.30.54.66:41671)
0808 07:11:11.483756 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.483768 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.483824 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.483981 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.483982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.484127 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:11.484188 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.484189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.485075 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.485075 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.485587 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.485589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.485895 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.485895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.485949 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.486369 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:11.486374 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:11.486649 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:11.488205 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.488205 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.488206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.488434 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.488651 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.488652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.488652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.488706 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.489022 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:11.489022 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:11.489058 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.489109 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.489182 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260808 07:11:11.489472 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.483622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:41671 (local address 127.30.54.126:45043)
0808 07:11:11.483664 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.483666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.483820 (+ 154us) server_negotiation.cc:407] Connection header received
0808 07:11:11.483882 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.483883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.483893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.483953 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:11.484241 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.484954 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.485657 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.485829 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.485880 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.486725 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:11.486726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:11.486727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:11.486732 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:11.488133 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.488519 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.488520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.488520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.488580 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.488772 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.488773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.488774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.488955 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:11.489026 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.489139 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.489331 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab.
Entry for principal oryx/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab.
I20260808 07:11:11.514184 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:45043
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab
--principal=oryx/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:32887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:11.596165 9762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:11.596400 9762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:11.596441 9762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:11.597576 9762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:11.597641 9762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:11.597679 9762 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:11:11.597738 9762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:11:11.599298 9762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:32887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab
--principal=oryx/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.54.126:45043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:11.599563 9762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:11.599815 9762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:11.604764 9770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:11.604785 9771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:11.604779 9773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:11.605024 9762 server_base.cc:1034] running on GCE node
I20260808 07:11:11.605252 9762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:11.605479 9762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:11.606642 9762 hybrid_clock.cc:648] HybridClock initialized: now 1786173071606621 us; error 29 us; skew 500 ppm
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:11.609091 9762 init.cc:377] Logged in from keytab as oryx/127.30.54.67@KRBTEST.COM (short username oryx)
I20260808 07:11:11.609531 9762 webserver.cc:460] Webserver started at http://127.30.54.67:42633/ using document root <none> and password file <none>
I20260808 07:11:11.609805 9762 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:11.609889 9762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:11.610036 9762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:11.611152 9762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "99b58453453b4f59a395eaff4859dde4"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "efcd4c4daf5331cf64f7423e466ecd18"
server_key_iv: "dd1efd0a2de7fb1466890a6920d327b3"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.611541 9762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "99b58453453b4f59a395eaff4859dde4"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "efcd4c4daf5331cf64f7423e466ecd18"
server_key_iv: "dd1efd0a2de7fb1466890a6920d327b3"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.612865 9762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:11.613520 9783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.613686 9762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:11.613756 9762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "99b58453453b4f59a395eaff4859dde4"
format_stamp: "Formatted at 2026-08-08 07:11:11 on dist-test-slave-znh6"
server_key: "efcd4c4daf5331cf64f7423e466ecd18"
server_key_iv: "dd1efd0a2de7fb1466890a6920d327b3"
server_key_version: "encryptionkey@0"
I20260808 07:11:11.613803 9762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:11.631728 9762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:11.634043 9762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:11.634157 9762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:11.634735 9762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:11.634806 9762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.634931 9762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:11.634975 9762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.641955 9762 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:45035
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:11:11.642035 9959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:45035 every 8 connection(s)
I20260808 07:11:11.642367 9762 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:11.645208 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.635072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:44647 (local address 127.30.54.126:45043)
0808 07:11:11.635149 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.635152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.635540 (+ 388us) server_negotiation.cc:407] Connection header received
0808 07:11:11.636100 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.636102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.636115 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.636166 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:11.636994 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.637892 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.639372 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.639728 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.639812 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.642622 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:11.642624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:11.642624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:11.642634 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:11.643994 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.644396 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.644397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.644397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.644468 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.644692 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.644693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.644693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.644802 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:11.644855 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.645076 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.645143 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:11:11.645382 9798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.635199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45043 (local address 127.30.54.67:44647)
0808 07:11:11.635400 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.635439 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.635995 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.636232 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.636236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.636566 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:11.636881 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.636889 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.637999 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.638000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.639250 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.639254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.639840 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.639842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.639984 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.640544 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:11.640554 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:11.642511 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:11.644074 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.644080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.644082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.644295 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.644538 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.644538 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.644539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.644632 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.644859 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:11.644863 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:11.644969 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.645050 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.645162 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
I20260808 07:11:11.646018 9960 heartbeater.cc:347] Connected to a master server at 127.30.54.126:45043
I20260808 07:11:11.646117 9960 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:11.646286 9960 heartbeater.cc:511] Master 127.30.54.126:45043 requested a full tablet report, sending...
I20260808 07:11:11.646694 9238 ts_manager.cc:194] Registered new tserver with Master: 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035)
I20260808 07:11:11.647234 9238 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:44647
I20260808 07:11:11.649456 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 9762
I20260808 07:11:11.649564 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:11.649753 30937 external_mini_cluster.cc:1428] Setting key c5e7666785791be54edd68146c44e732
I20260808 07:11:11.650624 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:11:11.652643 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.647110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45043 (local address 127.30.54.67:47381)
0808 07:11:11.647231 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.647241 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.647296 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.647420 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.647421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.647657 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:11.647725 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.647727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.648343 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.648344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.648845 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.648846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.649147 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.649148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.649191 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.649625 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:11.649631 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:11.649982 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:11.651740 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.651740 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.651741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.651909 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.652106 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:11.652107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:11.652107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:11.652157 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:11.652410 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:11.652411 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:11.652444 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.652506 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.652576 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260808 07:11:11.652763 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.647185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:47381 (local address 127.30.54.126:45043)
0808 07:11:11.647224 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.647226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.647282 (+ 56us) server_negotiation.cc:407] Connection header received
0808 07:11:11.647345 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.647346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.647356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.647415 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:11.647801 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.648257 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.648906 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.649086 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.649132 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.650055 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:11.650057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:11.650057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:11.650064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:11.651658 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.651981 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.651982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.651982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.652041 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.652216 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.652217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.652217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.652350 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:11.652417 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.652549 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.652629 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:11:11.673161 9322 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:11.667823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.30.54.126:45043)
0808 07:11:11.667884 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.667886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.667896 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:11.667954 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.667961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.667970 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.668034 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:11.668620 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.669104 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.669874 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.670069 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.670145 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.673108 (+ 2963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37144: BlockingRecv error: recv got EOF from 127.0.0.1:37144 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 07:11:11 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:11:11.680127 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.673971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.30.54.126:45043)
0808 07:11:11.674051 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.674053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.674066 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:11:11.674138 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.674139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.674147 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.674190 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:11.674619 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.675129 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.675774 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.675974 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.676044 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.677998 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:11.677999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:11.677999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:11.678005 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:11.679254 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.679560 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.679560 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.679561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.679609 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:11.679771 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:11.679771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:11.679771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:11.679861 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:11.679933 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.680015 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.680068 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:11:11.681651 9238 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:11.683423 9238 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:11.690380 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.685923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:43561 (local address 127.0.0.1:48826)
0808 07:11:11.686391 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.686405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.686466 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.686889 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.686890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.686892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:11.686953 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.686955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.687947 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.687948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.689344 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.689347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.690059 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.690060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.690146 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.690160 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.690243 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.690294 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":100,"threads_started":1}
I20260808 07:11:11.690413 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.686446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48826 (local address 127.30.54.65:43561)
0808 07:11:11.686618 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.686625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.686662 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:11:11.686711 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.686714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.686732 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.686878 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:11.687048 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.687837 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.689458 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.689972 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.690030 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.690081 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.690274 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.690311 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260808 07:11:11.691390 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.685923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:45035 (local address 127.0.0.1:45022)
0808 07:11:11.686083 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.686118 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.686184 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.686986 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.686987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.686990 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:11.687028 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.687029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.688213 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.688214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.689398 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.689400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.691147 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.691148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.691214 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.691219 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.691278 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.691317 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":62,"threads_started":1}
I20260808 07:11:11.691414 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.686197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.30.54.67:45035)
0808 07:11:11.686374 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.686378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.686386 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:11.686870 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.686877 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.686899 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.686972 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:11.687098 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.688119 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.690114 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.691053 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.691144 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.691214 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.691292 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.691327 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260808 07:11:11.691295 9451 tablet_service.cc:1467] Processing CreateTablet for tablet 72b0c0ccd1444068a67eec5d1707a19e (DEFAULT_TABLE table=test-table [id=800789ff834e4678aba23adf93f127f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:11.691691 9451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b0c0ccd1444068a67eec5d1707a19e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:11.691931 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.685922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:37829 (local address 127.0.0.1:57424)
0808 07:11:11.686079 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.686118 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.686184 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.686797 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.686799 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.686801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:11.686910 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.686913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.689487 (+ 2574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.689488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.690338 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.690341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.691705 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.691706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.691785 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.691790 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.691842 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.691880 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":104,"threads_started":1}
I20260808 07:11:11.692170 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.686197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57424 (local address 127.30.54.66:37829)
0808 07:11:11.686653 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.686657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.686666 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:11.686701 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.686703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.686719 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.686794 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:11.687011 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.688132 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.690452 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.691542 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.691645 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.691722 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.692055 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.692092 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":108,"threads_started":1}
I20260808 07:11:11.692236 9865 tablet_service.cc:1467] Processing CreateTablet for tablet 72b0c0ccd1444068a67eec5d1707a19e (DEFAULT_TABLE table=test-table [id=800789ff834e4678aba23adf93f127f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:11.692610 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b0c0ccd1444068a67eec5d1707a19e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:11.695459 9652 tablet_service.cc:1467] Processing CreateTablet for tablet 72b0c0ccd1444068a67eec5d1707a19e (DEFAULT_TABLE table=test-table [id=800789ff834e4678aba23adf93f127f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:11.695772 9652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b0c0ccd1444068a67eec5d1707a19e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:11.696401 10000 tablet_bootstrap.cc:492] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: Bootstrap starting.
I20260808 07:11:11.696719 10001 tablet_bootstrap.cc:492] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: Bootstrap starting.
I20260808 07:11:11.697043 10000 tablet_bootstrap.cc:654] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:11.697378 10000 log.cc:824] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:11.697407 10001 tablet_bootstrap.cc:654] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:11.697715 10001 log.cc:824] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:11.698283 10000 tablet_bootstrap.cc:492] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: No bootstrap required, opened a new log
I20260808 07:11:11.698355 10000 ts_tablet_manager.cc:1397] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:11.698498 10001 tablet_bootstrap.cc:492] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: No bootstrap required, opened a new log
I20260808 07:11:11.698546 10001 ts_tablet_manager.cc:1397] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:11.699609 10009 tablet_bootstrap.cc:492] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: Bootstrap starting.
I20260808 07:11:11.700335 10009 tablet_bootstrap.cc:654] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:11.700366 10000 raft_consensus.cc:348] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.700570 10000 raft_consensus.cc:374] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:11.700615 10000 raft_consensus.cc:729] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c2ec8add924a13ae05aa23a2984a70, State: Initialized, Role: FOLLOWER
I20260808 07:11:11.700703 10009 log.cc:824] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:11.700726 10000 consensus_queue.cc:260] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.700789 10001 raft_consensus.cc:348] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.700976 10001 raft_consensus.cc:374] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:11.700989 10000 ts_tablet_manager.cc:1428] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:11:11.701026 10001 raft_consensus.cc:729] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99b58453453b4f59a395eaff4859dde4, State: Initialized, Role: FOLLOWER
I20260808 07:11:11.701056 9546 heartbeater.cc:503] Master 127.30.54.126:45043 was elected leader, sending a full tablet report...
I20260808 07:11:11.701143 10001 consensus_queue.cc:260] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.701429 10001 ts_tablet_manager.cc:1428] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:11:11.701478 9960 heartbeater.cc:503] Master 127.30.54.126:45043 was elected leader, sending a full tablet report...
I20260808 07:11:11.701632 10009 tablet_bootstrap.cc:492] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: No bootstrap required, opened a new log
I20260808 07:11:11.701695 10009 ts_tablet_manager.cc:1397] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:11.703445 10009 raft_consensus.cc:348] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.703559 10009 raft_consensus.cc:374] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:11.703580 10009 raft_consensus.cc:729] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb44a37cb4864c66ac05348abba27b9e, State: Initialized, Role: FOLLOWER
I20260808 07:11:11.703712 10009 consensus_queue.cc:260] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.703972 10009 ts_tablet_manager.cc:1428] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:11:11.703991 9751 heartbeater.cc:503] Master 127.30.54.126:45043 was elected leader, sending a full tablet report...
W20260808 07:11:11.729990 9752 tablet.cc:2369] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:11.833578 9548 tablet.cc:2369] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:11.866012 10015 raft_consensus.cc:482] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:11.866132 10015 raft_consensus.cc:504] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.866533 10015 leader_election.cc:288] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829), 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561)
I20260808 07:11:11.870497 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.866872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:53943 (local address 127.30.54.66:37829)
0808 07:11:11.866931 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.866934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.867010 (+ 76us) server_negotiation.cc:407] Connection header received
0808 07:11:11.867062 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.867063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.867072 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.867126 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:11.867318 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.868018 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.869504 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.870075 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.870143 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.870152 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.870395 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.870433 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:11:11.870504 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.866780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:38703 (local address 127.30.54.65:43561)
0808 07:11:11.866852 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.866855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.866865 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:11.866915 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.866916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.866926 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.867004 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:11.867200 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.867974 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.869504 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.870059 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.870145 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.870152 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.870384 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.870426 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:11:11.870518 9798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.866699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:43561 (local address 127.30.54.67:38703)
0808 07:11:11.866757 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.866768 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.866855 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.867005 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.867006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.867008 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:11.867099 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.867101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.868083 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.868084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.869404 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.869407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.870169 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.870170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.870281 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.870290 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.870349 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.870403 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":103}
I20260808 07:11:11.870540 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.866718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:37829 (local address 127.30.54.67:53943)
0808 07:11:11.866907 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:11.866917 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:11.867005 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:11.867153 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:11.867154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:11.867155 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:11.867235 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.867236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.868092 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.868092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.869404 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:11.869408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.870169 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.870170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:11.870281 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.870290 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:11:11.870377 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:11:11.870407 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":43,"thread_start_us":117,"threads_started":1}
I20260808 07:11:11.870954 9480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b0c0ccd1444068a67eec5d1707a19e" candidate_uuid: "99b58453453b4f59a395eaff4859dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c2ec8add924a13ae05aa23a2984a70" is_pre_election: true
I20260808 07:11:11.871129 9480 raft_consensus.cc:2454] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99b58453453b4f59a395eaff4859dde4 in term 0.
I20260808 07:11:11.871112 9682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b0c0ccd1444068a67eec5d1707a19e" candidate_uuid: "99b58453453b4f59a395eaff4859dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb44a37cb4864c66ac05348abba27b9e" is_pre_election: true
I20260808 07:11:11.871294 9682 raft_consensus.cc:2454] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99b58453453b4f59a395eaff4859dde4 in term 0.
I20260808 07:11:11.871363 9787 leader_election.cc:302] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c2ec8add924a13ae05aa23a2984a70, 99b58453453b4f59a395eaff4859dde4; no voters:
I20260808 07:11:11.871564 10015 raft_consensus.cc:2781] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:11.871616 10015 raft_consensus.cc:482] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:11.871632 10015 raft_consensus.cc:3037] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:11.872681 10015 raft_consensus.cc:504] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.872884 10015 leader_election.cc:288] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [CANDIDATE]: Term 1 election: Requested vote from peers eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829), 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561)
I20260808 07:11:11.872992 9480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b0c0ccd1444068a67eec5d1707a19e" candidate_uuid: "99b58453453b4f59a395eaff4859dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c2ec8add924a13ae05aa23a2984a70"
I20260808 07:11:11.873041 9682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b0c0ccd1444068a67eec5d1707a19e" candidate_uuid: "99b58453453b4f59a395eaff4859dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb44a37cb4864c66ac05348abba27b9e"
I20260808 07:11:11.873085 9480 raft_consensus.cc:3037] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:11.873122 9682 raft_consensus.cc:3037] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:11.873852 9682 raft_consensus.cc:2454] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99b58453453b4f59a395eaff4859dde4 in term 1.
I20260808 07:11:11.873860 9480 raft_consensus.cc:2454] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99b58453453b4f59a395eaff4859dde4 in term 1.
I20260808 07:11:11.874058 9787 leader_election.cc:302] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c2ec8add924a13ae05aa23a2984a70, 99b58453453b4f59a395eaff4859dde4; no voters:
I20260808 07:11:11.874207 10015 raft_consensus.cc:2781] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:11.874339 10015 raft_consensus.cc:686] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 1 LEADER]: Becoming Leader. State: Replica: 99b58453453b4f59a395eaff4859dde4, State: Running, Role: LEADER
I20260808 07:11:11.874475 10015 consensus_queue.cc:237] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.875389 9237 catalog_manager.cc:5495] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99b58453453b4f59a395eaff4859dde4 (127.30.54.67). New cstate: current_term: 1 leader_uuid: "99b58453453b4f59a395eaff4859dde4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } health_report { overall_health: HEALTHY } } }
W20260808 07:11:11.893064 9961 tablet.cc:2369] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:11.915248 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:11.912806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.30.54.67:45035)
0808 07:11:11.912863 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:11.912865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:11.912875 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:11.912936 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:11.912937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:11.912963 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:11.913036 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:11.913207 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.913695 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.914469 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:11.914668 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:11.914746 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:11.914936 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:11.915031 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:11.915122 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:11.915145 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 07:11:11.915168 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:11:11.917104 9682 raft_consensus.cc:1261] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Refusing update from remote peer 99b58453453b4f59a395eaff4859dde4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:11.917104 9480 raft_consensus.cc:1261] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Refusing update from remote peer 99b58453453b4f59a395eaff4859dde4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:11.917454 10015 consensus_queue.cc:1035] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:11.917649 10020 consensus_queue.cc:1035] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:11.919999 10023 mvcc.cc:205] Tried to move back new op lower bound from 7316164902569447424 to 7316164902398423040. Current Snapshot: MvccSnapshot[applied={T|T < 7316164902569447424}]
I20260808 07:11:11.920150 10029 mvcc.cc:205] Tried to move back new op lower bound from 7316164902569447424 to 7316164902398423040. Current Snapshot: MvccSnapshot[applied={T|T < 7316164902569447424}]
I20260808 07:11:11.920545 10031 mvcc.cc:205] Tried to move back new op lower bound from 7316164902569447424 to 7316164902398423040. Current Snapshot: MvccSnapshot[applied={T|T < 7316164902569447424}]
I20260808 07:11:11.922662 9239 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:11.922740 9239 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:11.923548 9239 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105: Sending DeleteTablet for 3 replicas of tablet 72b0c0ccd1444068a67eec5d1707a19e
I20260808 07:11:11.923892 9865 tablet_service.cc:1514] Processing DeleteTablet for tablet 72b0c0ccd1444068a67eec5d1707a19e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:11 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:45022
I20260808 07:11:11.923985 9652 tablet_service.cc:1514] Processing DeleteTablet for tablet 72b0c0ccd1444068a67eec5d1707a19e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:11 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:57424
I20260808 07:11:11.924186 10044 tablet_replica.cc:330] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: stopping tablet replica
I20260808 07:11:11.924134 9239 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:11.924268 9239 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:11.924289 10044 raft_consensus.cc:2229] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:11.924320 9451 tablet_service.cc:1514] Processing DeleteTablet for tablet 72b0c0ccd1444068a67eec5d1707a19e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:11 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48826
I20260808 07:11:11.924396 10044 raft_consensus.cc:2258] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:11.924495 10047 tablet_replica.cc:330] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: stopping tablet replica
I20260808 07:11:11.924615 10047 raft_consensus.cc:2229] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:11.924718 10047 raft_consensus.cc:2258] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:11.924901 10044 ts_tablet_manager.cc:1905] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:11.925134 10047 ts_tablet_manager.cc:1905] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:11.925935 9451 tablet_service.cc:1467] Processing CreateTablet for tablet e085b80e79804a1eaa76b2295a3f8421 (DEFAULT_TABLE table=test-table [id=919c139f884148308058d55bb8fa0439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:11.926106 9451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e085b80e79804a1eaa76b2295a3f8421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:11.926697 10044 ts_tablet_manager.cc:1918] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:11.926767 10044 log.cc:1196] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/72b0c0ccd1444068a67eec5d1707a19e
I20260808 07:11:11.927030 10047 ts_tablet_manager.cc:1918] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:11.927099 10047 log.cc:1196] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/72b0c0ccd1444068a67eec5d1707a19e
I20260808 07:11:11.927119 10044 ts_tablet_manager.cc:1939] T 72b0c0ccd1444068a67eec5d1707a19e P eb44a37cb4864c66ac05348abba27b9e: Deleting consensus metadata
I20260808 07:11:11.927349 10047 ts_tablet_manager.cc:1939] T 72b0c0ccd1444068a67eec5d1707a19e P 36c2ec8add924a13ae05aa23a2984a70: Deleting consensus metadata
I20260808 07:11:11.927464 10000 tablet_bootstrap.cc:492] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: Bootstrap starting.
I20260808 07:11:11.927522 9220 catalog_manager.cc:4841] TS eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829): tablet 72b0c0ccd1444068a67eec5d1707a19e (table test-table [id=800789ff834e4678aba23adf93f127f8]) successfully deleted
I20260808 07:11:11.927547 9652 tablet_service.cc:1467] Processing CreateTablet for tablet e085b80e79804a1eaa76b2295a3f8421 (DEFAULT_TABLE table=test-table [id=919c139f884148308058d55bb8fa0439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:11.927652 9652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e085b80e79804a1eaa76b2295a3f8421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:11.927852 10000 tablet_bootstrap.cc:654] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:11.928606 10000 tablet_bootstrap.cc:492] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: No bootstrap required, opened a new log
I20260808 07:11:11.928637 10000 ts_tablet_manager.cc:1397] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:11.928910 10009 tablet_bootstrap.cc:492] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: Bootstrap starting.
I20260808 07:11:11.928934 9865 tablet_service.cc:1467] Processing CreateTablet for tablet e085b80e79804a1eaa76b2295a3f8421 (DEFAULT_TABLE table=test-table [id=919c139f884148308058d55bb8fa0439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:11.928931 10000 raft_consensus.cc:348] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.928995 10000 raft_consensus.cc:374] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:11.929021 10000 raft_consensus.cc:729] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c2ec8add924a13ae05aa23a2984a70, State: Initialized, Role: FOLLOWER
I20260808 07:11:11.929039 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e085b80e79804a1eaa76b2295a3f8421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:11.929072 10000 consensus_queue.cc:260] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.929183 10000 ts_tablet_manager.cc:1428] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:11.929311 10009 tablet_bootstrap.cc:654] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:11.929996 10009 tablet_bootstrap.cc:492] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: No bootstrap required, opened a new log
I20260808 07:11:11.930027 10009 ts_tablet_manager.cc:1397] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:11.930130 10009 raft_consensus.cc:348] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.930171 10009 raft_consensus.cc:374] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:11.930182 10009 raft_consensus.cc:729] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb44a37cb4864c66ac05348abba27b9e, State: Initialized, Role: FOLLOWER
I20260808 07:11:11.930241 10009 consensus_queue.cc:260] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.930312 10001 tablet_bootstrap.cc:492] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: Bootstrap starting.
I20260808 07:11:11.930334 10009 ts_tablet_manager.cc:1428] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:11.930549 9217 catalog_manager.cc:4841] TS 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561): tablet 72b0c0ccd1444068a67eec5d1707a19e (table test-table [id=800789ff834e4678aba23adf93f127f8]) successfully deleted
I20260808 07:11:11.930758 10001 tablet_bootstrap.cc:654] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:11.931071 10042 tablet_replica.cc:330] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: stopping tablet replica
I20260808 07:11:11.931241 10042 raft_consensus.cc:2229] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:11.931399 10042 raft_consensus.cc:2258] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:11.931602 10001 tablet_bootstrap.cc:492] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: No bootstrap required, opened a new log
I20260808 07:11:11.931655 10001 ts_tablet_manager.cc:1397] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:11.931813 10001 raft_consensus.cc:348] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.931906 10001 raft_consensus.cc:374] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:11.931929 10001 raft_consensus.cc:729] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99b58453453b4f59a395eaff4859dde4, State: Initialized, Role: FOLLOWER
I20260808 07:11:11.931969 10042 ts_tablet_manager.cc:1905] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:11.931969 10001 consensus_queue.cc:260] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:11.932116 10001 ts_tablet_manager.cc:1428] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:11.933712 10042 ts_tablet_manager.cc:1918] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:11.933771 10042 log.cc:1196] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/72b0c0ccd1444068a67eec5d1707a19e
I20260808 07:11:11.934012 10042 ts_tablet_manager.cc:1939] T 72b0c0ccd1444068a67eec5d1707a19e P 99b58453453b4f59a395eaff4859dde4: Deleting consensus metadata
I20260808 07:11:11.934435 9216 catalog_manager.cc:4841] TS 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035): tablet 72b0c0ccd1444068a67eec5d1707a19e (table test-table [id=800789ff834e4678aba23adf93f127f8]) successfully deleted
I20260808 07:11:12.155903 10010 raft_consensus.cc:482] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:12.156008 10010 raft_consensus.cc:504] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.156301 10010 leader_election.cc:288] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829), 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035)
I20260808 07:11:12.160172 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.156579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:60809 (local address 127.30.54.67:45035)
0808 07:11:12.156627 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.156629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.156795 (+ 166us) server_negotiation.cc:407] Connection header received
0808 07:11:12.156835 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.156836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.156848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.156910 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:12.157140 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.157743 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.159395 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.159759 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.159857 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.159868 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.160051 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.160092 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:11:12.160185 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.156577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:43919 (local address 127.30.54.66:37829)
0808 07:11:12.156627 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.156629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.156795 (+ 166us) server_negotiation.cc:407] Connection header received
0808 07:11:12.156835 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.156836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.156848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.156924 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:12.157139 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.157756 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.159440 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.159791 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.159884 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.159893 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.160062 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.160107 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:11:12.160224 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.156498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:37829 (local address 127.30.54.65:43919)
0808 07:11:12.156650 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.156662 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.156735 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.157014 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.157015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.157016 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:12.157075 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.157076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.157851 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.157852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.159301 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.159305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.159887 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.159889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.159981 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.159984 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.160054 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.160107 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":98,"thread_start_us":72,"threads_started":1}
I20260808 07:11:12.160403 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.156498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:45035 (local address 127.30.54.65:60809)
0808 07:11:12.156649 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.156662 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.156735 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.156935 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.156937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.156940 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:12.157033 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.157036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.157860 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.157861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.159301 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.159305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.159877 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.159879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.159967 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.159971 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.160044 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.160095 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":40,"thread_start_us":84,"threads_started":1}
I20260808 07:11:12.160491 9682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e085b80e79804a1eaa76b2295a3f8421" candidate_uuid: "36c2ec8add924a13ae05aa23a2984a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb44a37cb4864c66ac05348abba27b9e" is_pre_election: true
I20260808 07:11:12.160624 9682 raft_consensus.cc:2454] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36c2ec8add924a13ae05aa23a2984a70 in term 0.
I20260808 07:11:12.160737 9895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e085b80e79804a1eaa76b2295a3f8421" candidate_uuid: "36c2ec8add924a13ae05aa23a2984a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b58453453b4f59a395eaff4859dde4" is_pre_election: true
I20260808 07:11:12.160807 9391 leader_election.cc:302] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c2ec8add924a13ae05aa23a2984a70, eb44a37cb4864c66ac05348abba27b9e; no voters:
I20260808 07:11:12.160899 9895 raft_consensus.cc:2454] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36c2ec8add924a13ae05aa23a2984a70 in term 0.
I20260808 07:11:12.160949 10010 raft_consensus.cc:2781] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:12.161002 10010 raft_consensus.cc:482] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:12.161022 10010 raft_consensus.cc:3037] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.161710 10010 raft_consensus.cc:504] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.161813 10010 leader_election.cc:288] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [CANDIDATE]: Term 1 election: Requested vote from peers eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829), 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035)
I20260808 07:11:12.162037 9682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e085b80e79804a1eaa76b2295a3f8421" candidate_uuid: "36c2ec8add924a13ae05aa23a2984a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb44a37cb4864c66ac05348abba27b9e"
I20260808 07:11:12.162102 9682 raft_consensus.cc:3037] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.162737 9682 raft_consensus.cc:2454] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36c2ec8add924a13ae05aa23a2984a70 in term 1.
I20260808 07:11:12.162870 9895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e085b80e79804a1eaa76b2295a3f8421" candidate_uuid: "36c2ec8add924a13ae05aa23a2984a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b58453453b4f59a395eaff4859dde4"
I20260808 07:11:12.162930 9391 leader_election.cc:302] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c2ec8add924a13ae05aa23a2984a70, eb44a37cb4864c66ac05348abba27b9e; no voters:
I20260808 07:11:12.162969 9895 raft_consensus.cc:3037] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.163033 10010 raft_consensus.cc:2781] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:12.163188 10010 raft_consensus.cc:686] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 LEADER]: Becoming Leader. State: Replica: 36c2ec8add924a13ae05aa23a2984a70, State: Running, Role: LEADER
I20260808 07:11:12.163278 10010 consensus_queue.cc:237] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.163836 9895 raft_consensus.cc:2454] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36c2ec8add924a13ae05aa23a2984a70 in term 1.
I20260808 07:11:12.164088 9237 catalog_manager.cc:5495] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65). New cstate: current_term: 1 leader_uuid: "36c2ec8add924a13ae05aa23a2984a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:11:12 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173071, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.126@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:11:12.250727 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.243621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45043 (local address 127.0.0.1:37164)
0808 07:11:12.243870 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.243886 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.243993 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.244164 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.244165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.244491 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:12.244539 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.244541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.245218 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.245219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.245772 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.245774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.246152 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.246153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.246206 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.246749 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:12.246753 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:12.248204 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:12.249707 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:12.249714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:12.249716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:12.249948 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:12.250168 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:12.250169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:12.250169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:12.250228 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:12.250458 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:12.250464 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:12.250509 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.250563 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.250653 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":126,"threads_started":1}
I20260808 07:11:12.250845 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.243711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.30.54.126:45043)
0808 07:11:12.243903 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.243905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.243959 (+ 54us) server_negotiation.cc:407] Connection header received
0808 07:11:12.244058 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.244059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.244070 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.244160 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.244630 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.245137 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.245867 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.246066 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.246152 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.248288 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.248290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.248290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.248296 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.249617 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.250017 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.250018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.250018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.250070 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.250283 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.250284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.250284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.250401 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.250455 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.250595 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.250703 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":164,"thread_start_us":109,"threads_started":1}
I20260808 07:11:12.252746 9237 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:37164:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:11:12.252947 9237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:12.254458 9652 tablet_service.cc:1467] Processing CreateTablet for tablet 590a7d5ca495423d950906898267eab0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=621ac8d0cb83426aacf5f79a7faec6b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:12.254448 9865 tablet_service.cc:1467] Processing CreateTablet for tablet 590a7d5ca495423d950906898267eab0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=621ac8d0cb83426aacf5f79a7faec6b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:12.254621 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 590a7d5ca495423d950906898267eab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.254621 9652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 590a7d5ca495423d950906898267eab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.254593 9451 tablet_service.cc:1467] Processing CreateTablet for tablet 590a7d5ca495423d950906898267eab0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=621ac8d0cb83426aacf5f79a7faec6b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:11:12.254750 9451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 590a7d5ca495423d950906898267eab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.256012 10000 tablet_bootstrap.cc:492] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: Bootstrap starting.
I20260808 07:11:12.256218 10009 tablet_bootstrap.cc:492] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: Bootstrap starting.
I20260808 07:11:12.256232 10001 tablet_bootstrap.cc:492] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: Bootstrap starting.
I20260808 07:11:12.256479 10000 tablet_bootstrap.cc:654] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.256575 10001 tablet_bootstrap.cc:654] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.256599 10009 tablet_bootstrap.cc:654] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.257189 10001 tablet_bootstrap.cc:492] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: No bootstrap required, opened a new log
I20260808 07:11:12.257251 10001 ts_tablet_manager.cc:1397] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:12.257306 10009 tablet_bootstrap.cc:492] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: No bootstrap required, opened a new log
I20260808 07:11:12.257333 10000 tablet_bootstrap.cc:492] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: No bootstrap required, opened a new log
I20260808 07:11:12.257360 10009 ts_tablet_manager.cc:1397] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:12.257373 10000 ts_tablet_manager.cc:1397] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:12.257390 10001 raft_consensus.cc:348] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.257467 10001 raft_consensus.cc:374] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.257486 10001 raft_consensus.cc:729] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99b58453453b4f59a395eaff4859dde4, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.257507 10000 raft_consensus.cc:348] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.257529 10001 consensus_queue.cc:260] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.257567 10000 raft_consensus.cc:374] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.257589 10000 raft_consensus.cc:729] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c2ec8add924a13ae05aa23a2984a70, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.257620 10001 ts_tablet_manager.cc:1428] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.257535 10009 raft_consensus.cc:348] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.257632 10000 consensus_queue.cc:260] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.257659 10009 raft_consensus.cc:374] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.257684 10009 raft_consensus.cc:729] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb44a37cb4864c66ac05348abba27b9e, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.257725 10000 ts_tablet_manager.cc:1428] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.257728 10009 consensus_queue.cc:260] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.257762 10010 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.257817 10009 ts_tablet_manager.cc:1428] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:12.257826 10010 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:12.257965 10016 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.258075 10016 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:12.258565 10035 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.258654 10035 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:12.372258 10010 raft_consensus.cc:482] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:12.372390 10010 raft_consensus.cc:504] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.372573 10010 leader_election.cc:288] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829), 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035)
I20260808 07:11:12.372809 9682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "590a7d5ca495423d950906898267eab0" candidate_uuid: "36c2ec8add924a13ae05aa23a2984a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb44a37cb4864c66ac05348abba27b9e" is_pre_election: true
I20260808 07:11:12.372845 9895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "590a7d5ca495423d950906898267eab0" candidate_uuid: "36c2ec8add924a13ae05aa23a2984a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b58453453b4f59a395eaff4859dde4" is_pre_election: true
I20260808 07:11:12.372931 9682 raft_consensus.cc:2454] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36c2ec8add924a13ae05aa23a2984a70 in term 0.
I20260808 07:11:12.372926 9895 raft_consensus.cc:2454] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36c2ec8add924a13ae05aa23a2984a70 in term 0.
I20260808 07:11:12.373117 9386 leader_election.cc:302] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c2ec8add924a13ae05aa23a2984a70, 99b58453453b4f59a395eaff4859dde4; no voters:
I20260808 07:11:12.373242 10010 raft_consensus.cc:2781] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:12.373304 10010 raft_consensus.cc:482] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:12.373337 10010 raft_consensus.cc:3037] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.374058 10010 raft_consensus.cc:504] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.374228 10010 leader_election.cc:288] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [CANDIDATE]: Term 1 election: Requested vote from peers eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829), 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035)
I20260808 07:11:12.374382 9895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "590a7d5ca495423d950906898267eab0" candidate_uuid: "36c2ec8add924a13ae05aa23a2984a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b58453453b4f59a395eaff4859dde4"
I20260808 07:11:12.374454 9895 raft_consensus.cc:3037] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.374399 9682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "590a7d5ca495423d950906898267eab0" candidate_uuid: "36c2ec8add924a13ae05aa23a2984a70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb44a37cb4864c66ac05348abba27b9e"
I20260808 07:11:12.374488 9682 raft_consensus.cc:3037] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.375155 9895 raft_consensus.cc:2454] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36c2ec8add924a13ae05aa23a2984a70 in term 1.
I20260808 07:11:12.375229 9682 raft_consensus.cc:2454] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36c2ec8add924a13ae05aa23a2984a70 in term 1.
I20260808 07:11:12.375345 9386 leader_election.cc:302] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c2ec8add924a13ae05aa23a2984a70, 99b58453453b4f59a395eaff4859dde4; no voters:
I20260808 07:11:12.375453 10010 raft_consensus.cc:2781] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:12.375500 10010 raft_consensus.cc:686] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 LEADER]: Becoming Leader. State: Replica: 36c2ec8add924a13ae05aa23a2984a70, State: Running, Role: LEADER
I20260808 07:11:12.375553 10010 consensus_queue.cc:237] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.375911 10056 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: TxnStatusTablet state changed. Reason: New leader 36c2ec8add924a13ae05aa23a2984a70. Latest consensus state: current_term: 1 leader_uuid: "36c2ec8add924a13ae05aa23a2984a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.376039 10056 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:12.376245 10074 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:11:12.376207 9237 catalog_manager.cc:5495] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65). New cstate: current_term: 1 leader_uuid: "36c2ec8add924a13ae05aa23a2984a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:12.376385 10074 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:11:12.406451 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.402306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:43561 (local address 127.0.0.1:48840)
0808 07:11:12.402395 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.402406 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.402473 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.403277 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.403279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.403281 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:12.403353 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.403355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.404406 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.404407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.405322 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.405325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.405861 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.405862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.405924 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.405941 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.406262 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.406273 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.406321 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.406379 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 07:11:12.406451 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.402798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48840 (local address 127.30.54.65:43561)
0808 07:11:12.403061 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.403063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.403073 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:12.403122 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.403123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.403145 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.403225 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:12.403441 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.404300 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.405440 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.405762 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.405857 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.406088 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.406194 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.406277 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.406323 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.406345 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":130,"threads_started":1}
I20260808 07:11:12.407776 9895 raft_consensus.cc:1261] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Refusing update from remote peer 36c2ec8add924a13ae05aa23a2984a70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:12.408010 9682 raft_consensus.cc:1261] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Refusing update from remote peer 36c2ec8add924a13ae05aa23a2984a70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:12.408077 10056 consensus_queue.cc:1035] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:12.408259 10010 consensus_queue.cc:1035] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:12.408723 10035 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: TxnStatusTablet state changed. Reason: New leader 36c2ec8add924a13ae05aa23a2984a70. Latest consensus state: current_term: 1 leader_uuid: "36c2ec8add924a13ae05aa23a2984a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.408828 10035 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:12.409138 10016 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: TxnStatusTablet state changed. Reason: New leader 36c2ec8add924a13ae05aa23a2984a70. Latest consensus state: current_term: 1 leader_uuid: "36c2ec8add924a13ae05aa23a2984a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.409229 10016 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:12.409963 10010 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36c2ec8add924a13ae05aa23a2984a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.410044 10010 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:12.410132 10010 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36c2ec8add924a13ae05aa23a2984a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.410187 10010 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P 36c2ec8add924a13ae05aa23a2984a70: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:11:12.410223 10035 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36c2ec8add924a13ae05aa23a2984a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.410290 10035 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P 99b58453453b4f59a395eaff4859dde4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:12.410960 10016 tablet_replica.cc:439] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36c2ec8add924a13ae05aa23a2984a70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } } }
I20260808 07:11:12.411051 10016 tablet_replica.cc:442] T 590a7d5ca495423d950906898267eab0 P eb44a37cb4864c66ac05348abba27b9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:11:12.414927 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.412370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48846 (local address 127.30.54.65:43561)
0808 07:11:12.412427 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.412429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.412442 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:11:12.412503 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.412504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.412514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.412582 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:12.412697 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.413422 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.414211 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.414530 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.414604 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.414699 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.414730 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.414783 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.414838 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.414859 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:11:12.420439 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.417628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:34449 (local address 127.30.54.65:43561)
0808 07:11:12.417692 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.417694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.417795 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:11:12.417827 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.417828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.417837 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.417892 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:12.418013 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.418728 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.419542 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.419842 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.419915 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.420180 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.420212 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.420270 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.420327 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.420354 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":47}
I20260808 07:11:12.420585 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.417559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:43561 (local address 127.30.54.65:34449)
0808 07:11:12.417725 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.417735 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.417789 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.417901 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.417902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.417903 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:12.417961 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.417962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.418810 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.418811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.419462 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.419464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.419939 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.419940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.420002 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.420015 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.420283 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.420285 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.420325 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.420370 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":3,"thread_start_us":81,"threads_started":1}
I20260808 07:11:12.422377 9682 raft_consensus.cc:1261] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Refusing update from remote peer 36c2ec8add924a13ae05aa23a2984a70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:12.422449 9895 raft_consensus.cc:1261] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Refusing update from remote peer 36c2ec8add924a13ae05aa23a2984a70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:12.422616 10056 consensus_queue.cc:1035] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:12.422693 10010 consensus_queue.cc:1035] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:12.436056 9237 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:12.436167 9237 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:12.436817 9237 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105: Sending DeleteTablet for 3 replicas of tablet e085b80e79804a1eaa76b2295a3f8421
I20260808 07:11:12.437096 9451 tablet_service.cc:1514] Processing DeleteTablet for tablet e085b80e79804a1eaa76b2295a3f8421 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:12 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48826
I20260808 07:11:12.437187 9652 tablet_service.cc:1514] Processing DeleteTablet for tablet e085b80e79804a1eaa76b2295a3f8421 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:12 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:57424
I20260808 07:11:12.437278 10092 tablet_replica.cc:330] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: stopping tablet replica
I20260808 07:11:12.437340 10092 raft_consensus.cc:2229] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:12.437347 10093 tablet_replica.cc:330] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: stopping tablet replica
I20260808 07:11:12.437335 9865 tablet_service.cc:1514] Processing DeleteTablet for tablet e085b80e79804a1eaa76b2295a3f8421 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:12 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:45022
I20260808 07:11:12.437393 10093 raft_consensus.cc:2229] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:12.437419 10092 raft_consensus.cc:2258] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:12.437431 10093 raft_consensus.cc:2258] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:12.437674 10092 ts_tablet_manager.cc:1905] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:12.437677 10093 ts_tablet_manager.cc:1905] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:12.439652 10092 ts_tablet_manager.cc:1918] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:12.439674 10093 ts_tablet_manager.cc:1918] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:12.439718 10092 log.cc:1196] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/e085b80e79804a1eaa76b2295a3f8421
I20260808 07:11:12.439729 10093 log.cc:1196] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/e085b80e79804a1eaa76b2295a3f8421
I20260808 07:11:12.439975 10093 ts_tablet_manager.cc:1939] T e085b80e79804a1eaa76b2295a3f8421 P eb44a37cb4864c66ac05348abba27b9e: Deleting consensus metadata
I20260808 07:11:12.439983 10092 ts_tablet_manager.cc:1939] T e085b80e79804a1eaa76b2295a3f8421 P 36c2ec8add924a13ae05aa23a2984a70: Deleting consensus metadata
I20260808 07:11:12.440264 9217 catalog_manager.cc:4841] TS 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561): tablet e085b80e79804a1eaa76b2295a3f8421 (table test-table [id=919c139f884148308058d55bb8fa0439]) successfully deleted
I20260808 07:11:12.440282 9220 catalog_manager.cc:4841] TS eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829): tablet e085b80e79804a1eaa76b2295a3f8421 (table test-table [id=919c139f884148308058d55bb8fa0439]) successfully deleted
I20260808 07:11:12.441740 10096 tablet_replica.cc:330] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: stopping tablet replica
I20260808 07:11:12.441817 10096 raft_consensus.cc:2229] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:12.441861 10096 raft_consensus.cc:2258] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:12.442062 10096 ts_tablet_manager.cc:1905] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:12.443343 10096 ts_tablet_manager.cc:1918] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:12.443424 10096 log.cc:1196] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/e085b80e79804a1eaa76b2295a3f8421
I20260808 07:11:12.443639 10096 ts_tablet_manager.cc:1939] T e085b80e79804a1eaa76b2295a3f8421 P 99b58453453b4f59a395eaff4859dde4: Deleting consensus metadata
I20260808 07:11:12.444058 9216 catalog_manager.cc:4841] TS 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035): tablet e085b80e79804a1eaa76b2295a3f8421 (table test-table [id=919c139f884148308058d55bb8fa0439]) successfully deleted
Aug 08 07:11:12 dist-test-slave-znh6 krb5kdc[9170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:11:12 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:12 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:11:12.471427 10065 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:12.465796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.30.54.126:45043)
0808 07:11:12.465875 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.465877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.466009 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:11:12.466094 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.466095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.466106 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.466164 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.466720 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.467301 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.468051 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.468268 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.468353 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.471337 (+ 2984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37180: BlockingRecv error: recv got EOF from 127.0.0.1:37180 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 07:11:12 dist-test-slave-znh6 krb5kdc[9170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM
I20260808 07:11:12.479377 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.472403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37188 (local address 127.30.54.126:45043)
0808 07:11:12.472438 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.472440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.472562 (+ 122us) server_negotiation.cc:407] Connection header received
0808 07:11:12.472625 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.472626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.472635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.472684 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:12.473139 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.473589 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.474254 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.474447 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.474531 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.476825 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:12.476827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:12.476827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:12.476834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:12.478204 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.478542 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.478543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.478544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.478605 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:12.478805 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:12.478806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:12.478807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:12.479006 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:12.479105 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.479203 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.479280 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:11:12.483853 9237 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:12.484037 9237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:12.485478 9451 tablet_service.cc:1467] Processing CreateTablet for tablet 52d9966aa6e347a0ab440b7d3b6e084b (DEFAULT_TABLE table=test-table [id=fdc3ac09ac8d45509d14b933a92faa8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:12.485486 9652 tablet_service.cc:1467] Processing CreateTablet for tablet 52d9966aa6e347a0ab440b7d3b6e084b (DEFAULT_TABLE table=test-table [id=fdc3ac09ac8d45509d14b933a92faa8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:12.485544 9865 tablet_service.cc:1467] Processing CreateTablet for tablet 52d9966aa6e347a0ab440b7d3b6e084b (DEFAULT_TABLE table=test-table [id=fdc3ac09ac8d45509d14b933a92faa8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:12.485607 9451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52d9966aa6e347a0ab440b7d3b6e084b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.485607 9652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52d9966aa6e347a0ab440b7d3b6e084b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.485678 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52d9966aa6e347a0ab440b7d3b6e084b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.486940 10009 tablet_bootstrap.cc:492] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: Bootstrap starting.
I20260808 07:11:12.487120 10000 tablet_bootstrap.cc:492] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: Bootstrap starting.
I20260808 07:11:12.487324 10009 tablet_bootstrap.cc:654] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.488056 10009 tablet_bootstrap.cc:492] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: No bootstrap required, opened a new log
I20260808 07:11:12.488107 10009 ts_tablet_manager.cc:1397] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:12.488250 10009 raft_consensus.cc:348] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.488322 10009 raft_consensus.cc:374] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.488346 10009 raft_consensus.cc:729] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb44a37cb4864c66ac05348abba27b9e, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.488327 10001 tablet_bootstrap.cc:492] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: Bootstrap starting.
I20260808 07:11:12.488386 10009 consensus_queue.cc:260] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.488477 10009 ts_tablet_manager.cc:1428] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.488691 10001 tablet_bootstrap.cc:654] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.489416 10000 tablet_bootstrap.cc:654] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.489871 10001 tablet_bootstrap.cc:492] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: No bootstrap required, opened a new log
I20260808 07:11:12.489935 10001 ts_tablet_manager.cc:1397] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:11:12.490031 10000 tablet_bootstrap.cc:492] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: No bootstrap required, opened a new log
I20260808 07:11:12.490087 10000 ts_tablet_manager.cc:1397] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:11:12.490087 10001 raft_consensus.cc:348] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.490146 10001 raft_consensus.cc:374] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.490172 10001 raft_consensus.cc:729] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99b58453453b4f59a395eaff4859dde4, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.490212 10001 consensus_queue.cc:260] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.490293 10001 ts_tablet_manager.cc:1428] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:12.490289 10000 raft_consensus.cc:348] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.490368 10000 raft_consensus.cc:374] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.490399 10000 raft_consensus.cc:729] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c2ec8add924a13ae05aa23a2984a70, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.490445 10000 consensus_queue.cc:260] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.490547 10000 ts_tablet_manager.cc:1428] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.544847 10035 raft_consensus.cc:482] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:12.545035 10035 raft_consensus.cc:504] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.545217 10035 leader_election.cc:288] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561), eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829)
I20260808 07:11:12.545473 9682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52d9966aa6e347a0ab440b7d3b6e084b" candidate_uuid: "99b58453453b4f59a395eaff4859dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb44a37cb4864c66ac05348abba27b9e" is_pre_election: true
I20260808 07:11:12.545621 9682 raft_consensus.cc:2454] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99b58453453b4f59a395eaff4859dde4 in term 0.
I20260808 07:11:12.545593 9480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52d9966aa6e347a0ab440b7d3b6e084b" candidate_uuid: "99b58453453b4f59a395eaff4859dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c2ec8add924a13ae05aa23a2984a70" is_pre_election: true
I20260808 07:11:12.545698 9480 raft_consensus.cc:2454] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99b58453453b4f59a395eaff4859dde4 in term 0.
I20260808 07:11:12.545882 9792 leader_election.cc:302] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99b58453453b4f59a395eaff4859dde4, eb44a37cb4864c66ac05348abba27b9e; no voters:
I20260808 07:11:12.545977 10035 raft_consensus.cc:2781] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:12.546012 10035 raft_consensus.cc:482] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:12.546064 10035 raft_consensus.cc:3037] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.547088 10035 raft_consensus.cc:504] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.547230 10035 leader_election.cc:288] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [CANDIDATE]: Term 1 election: Requested vote from peers 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561), eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829)
I20260808 07:11:12.547386 9480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52d9966aa6e347a0ab440b7d3b6e084b" candidate_uuid: "99b58453453b4f59a395eaff4859dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c2ec8add924a13ae05aa23a2984a70"
I20260808 07:11:12.547385 9682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52d9966aa6e347a0ab440b7d3b6e084b" candidate_uuid: "99b58453453b4f59a395eaff4859dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb44a37cb4864c66ac05348abba27b9e"
I20260808 07:11:12.547463 9480 raft_consensus.cc:3037] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.547489 9682 raft_consensus.cc:3037] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.548103 9682 raft_consensus.cc:2454] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99b58453453b4f59a395eaff4859dde4 in term 1.
I20260808 07:11:12.548239 9480 raft_consensus.cc:2454] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99b58453453b4f59a395eaff4859dde4 in term 1.
I20260808 07:11:12.548276 9792 leader_election.cc:302] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99b58453453b4f59a395eaff4859dde4, eb44a37cb4864c66ac05348abba27b9e; no voters:
I20260808 07:11:12.548372 10035 raft_consensus.cc:2781] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:12.548497 10035 raft_consensus.cc:686] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 1 LEADER]: Becoming Leader. State: Replica: 99b58453453b4f59a395eaff4859dde4, State: Running, Role: LEADER
I20260808 07:11:12.548558 10035 consensus_queue.cc:237] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.549187 9239 catalog_manager.cc:5495] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99b58453453b4f59a395eaff4859dde4 (127.30.54.67). New cstate: current_term: 1 leader_uuid: "99b58453453b4f59a395eaff4859dde4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } health_report { overall_health: HEALTHY } } }
I20260808 07:11:12.564304 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.561862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.30.54.67:45035)
0808 07:11:12.561904 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.561906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.561915 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:12.561990 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.561991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.562001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.562059 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:12.562201 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.562676 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.563573 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.563820 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.563911 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.564069 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.564102 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.564149 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.564206 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.564227 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:12.565104 9682 raft_consensus.cc:1261] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Refusing update from remote peer 99b58453453b4f59a395eaff4859dde4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:12.565187 9480 raft_consensus.cc:1261] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Refusing update from remote peer 99b58453453b4f59a395eaff4859dde4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:12.565296 10118 consensus_queue.cc:1035] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:12.565528 10118 consensus_queue.cc:1035] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:12.568948 9238 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:12.569042 9238 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:12.569676 9238 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105: Sending DeleteTablet for 3 replicas of tablet 52d9966aa6e347a0ab440b7d3b6e084b
I20260808 07:11:12.570024 9865 tablet_service.cc:1514] Processing DeleteTablet for tablet 52d9966aa6e347a0ab440b7d3b6e084b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:12 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:45022
I20260808 07:11:12.570030 9652 tablet_service.cc:1514] Processing DeleteTablet for tablet 52d9966aa6e347a0ab440b7d3b6e084b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:12 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:57424
I20260808 07:11:12.570066 9451 tablet_service.cc:1514] Processing DeleteTablet for tablet 52d9966aa6e347a0ab440b7d3b6e084b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:12 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48826
I20260808 07:11:12.570111 10096 tablet_replica.cc:330] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: stopping tablet replica
I20260808 07:11:12.570148 10093 tablet_replica.cc:330] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: stopping tablet replica
I20260808 07:11:12.570159 10096 raft_consensus.cc:2229] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:12.570204 10093 raft_consensus.cc:2229] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:12.570211 10096 raft_consensus.cc:2258] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:12.570252 10093 raft_consensus.cc:2258] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:12.570286 10092 tablet_replica.cc:330] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: stopping tablet replica
I20260808 07:11:12.570340 10092 raft_consensus.cc:2229] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:12.570377 10092 raft_consensus.cc:2258] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:12.570611 10093 ts_tablet_manager.cc:1905] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:12.570557 9238 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:12.570686 9238 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:12.571086 10096 ts_tablet_manager.cc:1905] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:12.572214 10093 ts_tablet_manager.cc:1918] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:12.572285 10093 log.cc:1196] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/52d9966aa6e347a0ab440b7d3b6e084b
I20260808 07:11:12.572512 10093 ts_tablet_manager.cc:1939] T 52d9966aa6e347a0ab440b7d3b6e084b P eb44a37cb4864c66ac05348abba27b9e: Deleting consensus metadata
I20260808 07:11:12.572588 10096 ts_tablet_manager.cc:1918] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:12.572654 10096 log.cc:1196] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/52d9966aa6e347a0ab440b7d3b6e084b
I20260808 07:11:12.572795 9220 catalog_manager.cc:4841] TS eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829): tablet 52d9966aa6e347a0ab440b7d3b6e084b (table test-table [id=fdc3ac09ac8d45509d14b933a92faa8c]) successfully deleted
I20260808 07:11:12.572889 10096 ts_tablet_manager.cc:1939] T 52d9966aa6e347a0ab440b7d3b6e084b P 99b58453453b4f59a395eaff4859dde4: Deleting consensus metadata
I20260808 07:11:12.573318 10092 ts_tablet_manager.cc:1905] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:12.574136 9451 tablet_service.cc:1467] Processing CreateTablet for tablet fc18e43030aa47779a079569f7fca595 (DEFAULT_TABLE table=test-table [id=adf978c8663b4a469d91c4fef01be9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:12.574265 9451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc18e43030aa47779a079569f7fca595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.574787 10092 ts_tablet_manager.cc:1918] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:12.574869 10092 log.cc:1196] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/52d9966aa6e347a0ab440b7d3b6e084b
I20260808 07:11:12.575093 10092 ts_tablet_manager.cc:1939] T 52d9966aa6e347a0ab440b7d3b6e084b P 36c2ec8add924a13ae05aa23a2984a70: Deleting consensus metadata
I20260808 07:11:12.575341 9216 catalog_manager.cc:4841] TS 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035): tablet 52d9966aa6e347a0ab440b7d3b6e084b (table test-table [id=fdc3ac09ac8d45509d14b933a92faa8c]) successfully deleted
I20260808 07:11:12.575412 10000 tablet_bootstrap.cc:492] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70: Bootstrap starting.
I20260808 07:11:12.575450 9865 tablet_service.cc:1467] Processing CreateTablet for tablet fc18e43030aa47779a079569f7fca595 (DEFAULT_TABLE table=test-table [id=adf978c8663b4a469d91c4fef01be9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:12.575457 9217 catalog_manager.cc:4841] TS 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561): tablet 52d9966aa6e347a0ab440b7d3b6e084b (table test-table [id=fdc3ac09ac8d45509d14b933a92faa8c]) successfully deleted
I20260808 07:11:12.575542 9865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc18e43030aa47779a079569f7fca595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.575729 10000 tablet_bootstrap.cc:654] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.576083 9652 tablet_service.cc:1467] Processing CreateTablet for tablet fc18e43030aa47779a079569f7fca595 (DEFAULT_TABLE table=test-table [id=adf978c8663b4a469d91c4fef01be9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:12.576192 9652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc18e43030aa47779a079569f7fca595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:12.576669 10001 tablet_bootstrap.cc:492] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: Bootstrap starting.
I20260808 07:11:12.576977 10001 tablet_bootstrap.cc:654] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.577286 10009 tablet_bootstrap.cc:492] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: Bootstrap starting.
I20260808 07:11:12.577497 10001 tablet_bootstrap.cc:492] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: No bootstrap required, opened a new log
I20260808 07:11:12.577540 10001 ts_tablet_manager.cc:1397] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:12.577564 10000 tablet_bootstrap.cc:492] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70: No bootstrap required, opened a new log
I20260808 07:11:12.577615 10000 ts_tablet_manager.cc:1397] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:11:12.577633 10001 raft_consensus.cc:348] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.577692 10001 raft_consensus.cc:374] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.577713 10001 raft_consensus.cc:729] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99b58453453b4f59a395eaff4859dde4, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.577721 10000 raft_consensus.cc:348] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.577751 10001 consensus_queue.cc:260] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.577796 10000 raft_consensus.cc:374] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.577816 10000 raft_consensus.cc:729] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c2ec8add924a13ae05aa23a2984a70, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.577844 10001 ts_tablet_manager.cc:1428] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.577858 10000 consensus_queue.cc:260] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.577955 10000 ts_tablet_manager.cc:1428] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:12.577989 10009 tablet_bootstrap.cc:654] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:12.578666 10009 tablet_bootstrap.cc:492] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: No bootstrap required, opened a new log
I20260808 07:11:12.578735 10009 ts_tablet_manager.cc:1397] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:12.578917 10009 raft_consensus.cc:348] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.578992 10009 raft_consensus.cc:374] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:12.579025 10009 raft_consensus.cc:729] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb44a37cb4864c66ac05348abba27b9e, State: Initialized, Role: FOLLOWER
I20260808 07:11:12.579075 10009 consensus_queue.cc:260] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.579167 10009 ts_tablet_manager.cc:1428] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:12.691484 10016 raft_consensus.cc:482] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:12.691599 10016 raft_consensus.cc:504] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.691963 10016 leader_election.cc:288] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561), 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035)
I20260808 07:11:12.695084 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.692258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:41867 (local address 127.30.54.67:45035)
0808 07:11:12.692310 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.692311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.692341 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:11:12.692441 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.692442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.692456 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.692528 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:12.692680 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.693169 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.694345 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.694700 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.694793 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.694803 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.694984 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.695019 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:11:12.695150 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.692157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:45035 (local address 127.30.54.66:41867)
0808 07:11:12.692271 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.692285 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.692360 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.692534 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.692535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.692536 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:12.692607 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.692609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.693278 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.693279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.694251 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.694254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.694795 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.694796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.694905 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.694908 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.694976 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.695014 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260808 07:11:12.695143 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.692312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:60969 (local address 127.30.54.65:43561)
0808 07:11:12.692389 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.692391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.692400 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:12.692429 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.692430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.692439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.692510 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:12.692707 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.693173 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.694349 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.694731 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.694812 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.694821 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.695030 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.695072 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:11:12.695165 10132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.692157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:43561 (local address 127.30.54.66:60969)
0808 07:11:12.692290 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.692301 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.692357 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.692537 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.692537 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.692540 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:12.692624 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.692626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.693272 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.693273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.694260 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.694263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.694887 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.694888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.694952 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.694955 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.695024 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.695064 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":5,"thread_start_us":68,"threads_started":1}
I20260808 07:11:12.695385 9480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc18e43030aa47779a079569f7fca595" candidate_uuid: "eb44a37cb4864c66ac05348abba27b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c2ec8add924a13ae05aa23a2984a70" is_pre_election: true
I20260808 07:11:12.695477 9480 raft_consensus.cc:2454] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb44a37cb4864c66ac05348abba27b9e in term 0.
I20260808 07:11:12.695453 9895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc18e43030aa47779a079569f7fca595" candidate_uuid: "eb44a37cb4864c66ac05348abba27b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b58453453b4f59a395eaff4859dde4" is_pre_election: true
I20260808 07:11:12.695526 9895 raft_consensus.cc:2454] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb44a37cb4864c66ac05348abba27b9e in term 0.
I20260808 07:11:12.695652 9584 leader_election.cc:302] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c2ec8add924a13ae05aa23a2984a70, eb44a37cb4864c66ac05348abba27b9e; no voters:
I20260808 07:11:12.695818 10016 raft_consensus.cc:2781] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:12.695869 10016 raft_consensus.cc:482] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:12.695894 10016 raft_consensus.cc:3037] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.696534 10016 raft_consensus.cc:504] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.696671 10016 leader_election.cc:288] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [CANDIDATE]: Term 1 election: Requested vote from peers 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561), 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035)
I20260808 07:11:12.696821 9895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc18e43030aa47779a079569f7fca595" candidate_uuid: "eb44a37cb4864c66ac05348abba27b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b58453453b4f59a395eaff4859dde4"
I20260808 07:11:12.696820 9480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc18e43030aa47779a079569f7fca595" candidate_uuid: "eb44a37cb4864c66ac05348abba27b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c2ec8add924a13ae05aa23a2984a70"
I20260808 07:11:12.696898 9895 raft_consensus.cc:3037] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.696898 9480 raft_consensus.cc:3037] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:12.697674 9480 raft_consensus.cc:2454] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb44a37cb4864c66ac05348abba27b9e in term 1.
I20260808 07:11:12.697674 9895 raft_consensus.cc:2454] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb44a37cb4864c66ac05348abba27b9e in term 1.
I20260808 07:11:12.697885 9583 leader_election.cc:302] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99b58453453b4f59a395eaff4859dde4, eb44a37cb4864c66ac05348abba27b9e; no voters:
I20260808 07:11:12.698017 10016 raft_consensus.cc:2781] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:12.698127 10016 raft_consensus.cc:686] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 1 LEADER]: Becoming Leader. State: Replica: eb44a37cb4864c66ac05348abba27b9e, State: Running, Role: LEADER
I20260808 07:11:12.698217 10016 consensus_queue.cc:237] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } }
I20260808 07:11:12.698977 9238 catalog_manager.cc:5495] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e reported cstate change: term changed from 0 to 1, leader changed from <none> to eb44a37cb4864c66ac05348abba27b9e (127.30.54.66). New cstate: current_term: 1 leader_uuid: "eb44a37cb4864c66ac05348abba27b9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb44a37cb4864c66ac05348abba27b9e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 37829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:12.728500 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.726123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.30.54.66:37829)
0808 07:11:12.726297 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.726299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.726307 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:12.726350 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.726351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.726373 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.726449 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:12.726534 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.727108 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.727788 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.728004 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.728068 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.728199 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.728305 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.728362 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.728404 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.728420 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
I20260808 07:11:12.733392 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.730882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:37103 (local address 127.30.54.65:43561)
0808 07:11:12.730940 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.730942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.731056 (+ 114us) server_negotiation.cc:407] Connection header received
0808 07:11:12.731101 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.731102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.731113 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.731155 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:12.731417 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.731874 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.732670 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.732847 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.732902 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.733108 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.733147 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.733188 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.733320 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.733343 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:11:12.733407 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.730787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:43561 (local address 127.30.54.66:37103)
0808 07:11:12.730977 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.730987 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.731050 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.731166 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.731167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.731168 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:12.731222 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.731223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.731970 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.731971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.732599 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.732601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.732939 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.732940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.732999 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.733011 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.733221 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.733223 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.733301 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.733340 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":129,"threads_started":1}
I20260808 07:11:12.735503 9480 raft_consensus.cc:1261] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Refusing update from remote peer eb44a37cb4864c66ac05348abba27b9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:12.735559 9895 raft_consensus.cc:1261] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Refusing update from remote peer eb44a37cb4864c66ac05348abba27b9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:12.735808 10135 consensus_queue.cc:1035] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "36c2ec8add924a13ae05aa23a2984a70" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 43561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:12.735940 10016 consensus_queue.cc:1035] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99b58453453b4f59a395eaff4859dde4" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:12.742446 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.740356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:36423 (local address 127.30.54.66:37829)
0808 07:11:12.740394 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:12.740396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:12.740404 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:12.740425 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:12.740426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:12.740435 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:12.740492 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:12.740598 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.741053 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.741746 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.741917 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.741969 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.742183 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.742215 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.742269 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:12.742361 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 07:11:12.742383 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:11:12.742482 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.740180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:37829 (local address 127.30.54.65:36423)
0808 07:11:12.740237 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.740247 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:12.740283 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:12.740495 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:12.740496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:12.740497 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:11:12.740535 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.740536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.741143 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.741144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.741666 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:12.741668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:12.742039 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:12.742040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:12.742099 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:12.742105 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:11:12.742288 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:11:12.742290 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:12.742353 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 07:11:12.742406 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:11:12.749495 9238 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:12.749606 9238 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:12.750270 9238 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b774c867ab84a4a8dec70240f335105: Sending DeleteTablet for 3 replicas of tablet fc18e43030aa47779a079569f7fca595
I20260808 07:11:12.750552 9652 tablet_service.cc:1514] Processing DeleteTablet for tablet fc18e43030aa47779a079569f7fca595 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:12 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:57424
I20260808 07:11:12.750648 10093 tablet_replica.cc:330] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: stopping tablet replica
I20260808 07:11:12.750628 9451 tablet_service.cc:1514] Processing DeleteTablet for tablet fc18e43030aa47779a079569f7fca595 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:12 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48826
I20260808 07:11:12.750703 10093 raft_consensus.cc:2229] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:12.750715 9865 tablet_service.cc:1514] Processing DeleteTablet for tablet fc18e43030aa47779a079569f7fca595 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:12 UTC) from {username='oryx', principal='oryx/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:45022
I20260808 07:11:12.750788 10093 raft_consensus.cc:2258] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:12.750797 10092 tablet_replica.cc:330] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70: stopping tablet replica
I20260808 07:11:12.750885 10092 raft_consensus.cc:2229] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:12.750927 10092 raft_consensus.cc:2258] T fc18e43030aa47779a079569f7fca595 P 36c2ec8add924a13ae05aa23a2984a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:12.751000 10096 tablet_replica.cc:330] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: stopping tablet replica
I20260808 07:11:12.751055 10096 raft_consensus.cc:2229] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:12.751093 10096 raft_consensus.cc:2258] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:12.751127 10093 ts_tablet_manager.cc:1905] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:12.751456 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 9356
I20260808 07:11:12.751588 10096 ts_tablet_manager.cc:1905] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:12.753170 10096 ts_tablet_manager.cc:1918] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:12.753218 10096 log.cc:1196] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/fc18e43030aa47779a079569f7fca595
I20260808 07:11:12.753400 10096 ts_tablet_manager.cc:1939] T fc18e43030aa47779a079569f7fca595 P 99b58453453b4f59a395eaff4859dde4: Deleting consensus metadata
I20260808 07:11:12.754143 10093 ts_tablet_manager.cc:1918] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:11:12.754191 10093 log.cc:1196] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/fc18e43030aa47779a079569f7fca595
I20260808 07:11:12.754272 9216 catalog_manager.cc:4841] TS 99b58453453b4f59a395eaff4859dde4 (127.30.54.67:45035): tablet fc18e43030aa47779a079569f7fca595 (table test-table [id=adf978c8663b4a469d91c4fef01be9f9]) successfully deleted
I20260808 07:11:12.754438 10093 ts_tablet_manager.cc:1939] T fc18e43030aa47779a079569f7fca595 P eb44a37cb4864c66ac05348abba27b9e: Deleting consensus metadata
I20260808 07:11:12.754709 9220 catalog_manager.cc:4841] TS eb44a37cb4864c66ac05348abba27b9e (127.30.54.66:37829): tablet fc18e43030aa47779a079569f7fca595 (table test-table [id=adf978c8663b4a469d91c4fef01be9f9]) successfully deleted
W20260808 07:11:12.769282 9217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:43561 (error 108)
I20260808 07:11:12.769500 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 9564
I20260808 07:11:12.773463 10157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:12.771957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:43561 (local address 127.0.0.1:48850)
0808 07:11:12.773232 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:12.773307 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:43561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1215,"thread_start_us":1025,"threads_started":1}
W20260808 07:11:12.774202 9217 catalog_manager.cc:4584] TS 36c2ec8add924a13ae05aa23a2984a70 (127.30.54.65:43561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc18e43030aa47779a079569f7fca595: Network error: Client connection negotiation failed: client connection to 127.30.54.65:43561: connect: Connection refused (error 111)
I20260808 07:11:12.782312 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 9762
I20260808 07:11:12.793272 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 9190
2026-08-08T07:11:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3148 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:12 dist-test-slave-znh6 krb5kdc[10161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:12 dist-test-slave-znh6 krb5kdc[10161](info): set up 2 sockets
Aug 08 07:11:12 dist-test-slave-znh6 krb5kdc[10161](info): commencing operation
krb5kdc: starting...
W20260808 07:11:14.004760 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:11:14 dist-test-slave-znh6 krb5kdc[10161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:11:14.124019 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34995
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:34995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:14.204326 10179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:14.204547 10179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:14.204586 10179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:14.205767 10179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:14.205833 10179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:14.205857 10179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:14.205880 10179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:14.205909 10179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:14.207412 10179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:34995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:14.207674 10179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:14.207888 10179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:14.212718 10191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:14.212674 10190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:14.212769 10179 server_base.cc:1034] running on GCE node
W20260808 07:11:14.212680 10193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:14.213089 10179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:14.213420 10179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:14.214570 10179 hybrid_clock.cc:648] HybridClock initialized: now 1786173074214554 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T07:11:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1421 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:14 dist-test-slave-znh6 krb5kdc[10199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:14 dist-test-slave-znh6 krb5kdc[10199](info): set up 2 sockets
Aug 08 07:11:14 dist-test-slave-znh6 krb5kdc[10199](info): commencing operation
krb5kdc: starting...
W20260808 07:11:15.454109 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:11:15 dist-test-slave-znh6 krb5kdc[10199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:11:15.573990 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34041
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:34041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:15.654768 10215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:15.654984 10215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:15.655009 10215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:15.656143 10215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:15.656186 10215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:15.656200 10215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:15.656213 10215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:15.656225 10215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:15.657693 10215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:34041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:15.657938 10215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:15.658159 10215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:15.663093 10222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:15.663123 10227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:15.663123 10224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:15.663448 10215 server_base.cc:1034] running on GCE node
I20260808 07:11:15.663707 10215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:15.664028 10215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:15.665215 10215 hybrid_clock.cc:648] HybridClock initialized: now 1786173075665212 us; error 42 us; skew 500 ppm
Aug 08 07:11:15 dist-test-slave-znh6 krb5kdc[10199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173075, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:15.671056 10215 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:11:15.671509 10215 webserver.cc:460] Webserver started at http://127.30.54.126:37703/ using document root <none> and password file <none>
I20260808 07:11:15.671756 10215 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:15.671828 10215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:15.671983 10215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:15.672996 10215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "9943289736554a7dab7ae6debb82b62d"
format_stamp: "Formatted at 2026-08-08 07:11:15 on dist-test-slave-znh6"
server_key: "6191b732b2f404bbcdb69ba325e90fae"
server_key_iv: "51b5226bcce66f9313bf6e56673e8236"
server_key_version: "encryptionkey@0"
I20260808 07:11:15.673367 10215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "9943289736554a7dab7ae6debb82b62d"
format_stamp: "Formatted at 2026-08-08 07:11:15 on dist-test-slave-znh6"
server_key: "6191b732b2f404bbcdb69ba325e90fae"
server_key_iv: "51b5226bcce66f9313bf6e56673e8236"
server_key_version: "encryptionkey@0"
I20260808 07:11:15.674752 10215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:11:15.675845 10236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:15.676194 10215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:15.676291 10215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "9943289736554a7dab7ae6debb82b62d"
format_stamp: "Formatted at 2026-08-08 07:11:15 on dist-test-slave-znh6"
server_key: "6191b732b2f404bbcdb69ba325e90fae"
server_key_iv: "51b5226bcce66f9313bf6e56673e8236"
server_key_version: "encryptionkey@0"
I20260808 07:11:15.676357 10215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:15.692909 10215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:15.694721 10215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:15.694926 10215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:15.698726 10215 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:34041
I20260808 07:11:15.698812 10318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:34041 every 8 connection(s)
I20260808 07:11:15.699165 10215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:15.700055 10319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:15.702193 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: Bootstrap starting.
I20260808 07:11:15.702705 10319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:15.703018 10319 log.cc:824] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:15.703688 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: No bootstrap required, opened a new log
I20260808 07:11:15.705202 10319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } }
I20260808 07:11:15.705328 10319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:15.705350 10319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9943289736554a7dab7ae6debb82b62d, State: Initialized, Role: FOLLOWER
I20260808 07:11:15.705489 10319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } }
I20260808 07:11:15.705572 10319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:15.705595 10319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:15.705626 10319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:15.706074 10319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } }
I20260808 07:11:15.706156 10319 leader_election.cc:302] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9943289736554a7dab7ae6debb82b62d; no voters:
I20260808 07:11:15.706271 10319 leader_election.cc:288] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:15.706323 10325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:15.706494 10325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 1 LEADER]: Becoming Leader. State: Replica: 9943289736554a7dab7ae6debb82b62d, State: Running, Role: LEADER
I20260808 07:11:15.706542 10319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:15.706627 10325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } }
I20260808 07:11:15.707405 10326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9943289736554a7dab7ae6debb82b62d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } } }
I20260808 07:11:15.707414 10327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9943289736554a7dab7ae6debb82b62d. Latest consensus state: current_term: 1 leader_uuid: "9943289736554a7dab7ae6debb82b62d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } } }
I20260808 07:11:15.707520 10326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:15.707552 10327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:15.707732 10341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:15.708317 10341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:15.708482 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 10215
I20260808 07:11:15.708599 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:11:15.708806 30937 external_mini_cluster.cc:1428] Setting key 4bbb9d1898de2e91e79cb1890fc32584
I20260808 07:11:15.709885 10341 catalog_manager.cc:1347] Generated new cluster ID: d7a3f817228449a09c88d4f838f96ca5
I20260808 07:11:15.709944 10341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:11:15 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:11:15.720152 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.709592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39214 (local address 127.30.54.126:34041)
0808 07:11:15.709777 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.709787 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.709820 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:11:15.710999 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.711004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.711047 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.711189 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:15.711827 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.712746 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.713499 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.713918 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.714036 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.716312 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:15.716315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:15.716315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:15.716333 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:15.718458 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.719058 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.719060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.719061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.719139 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.719350 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.719352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.719352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.719590 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:15.719766 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.719839 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.719980 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260808 07:11:15.723727 10341 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:15.724340 10341 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:15.729523 10341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: Generated new TSK 0
I20260808 07:11:15.729684 10341 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:11:15.764048 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34041
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:15.872319 10367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:15.872517 10367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:15.872540 10367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:15.873695 10367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:15.873740 10367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:15.873766 10367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:11:15.875244 10367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:15.875476 10367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:15.875725 10367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:15.880841 10380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:15.880900 10377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:15.880952 10367 server_base.cc:1034] running on GCE node
W20260808 07:11:15.880925 10374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:15.881263 10367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:15.881465 10367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:15.882612 10367 hybrid_clock.cc:648] HybridClock initialized: now 1786173075882604 us; error 31 us; skew 500 ppm
Aug 08 07:11:15 dist-test-slave-znh6 krb5kdc[10199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173075, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:15.885183 10367 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:11:15.885637 10367 webserver.cc:460] Webserver started at http://127.30.54.65:46275/ using document root <none> and password file <none>
I20260808 07:11:15.885867 10367 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:15.885936 10367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:15.886090 10367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:15.887215 10367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "2731c7e0c2ae4728af3d1292738e7d5f"
format_stamp: "Formatted at 2026-08-08 07:11:15 on dist-test-slave-znh6"
server_key: "c972c39dbc94154b9e1fbf2abeb5b676"
server_key_iv: "b0aba0b661e57823db74978632a72ce1"
server_key_version: "encryptionkey@0"
I20260808 07:11:15.887584 10367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "2731c7e0c2ae4728af3d1292738e7d5f"
format_stamp: "Formatted at 2026-08-08 07:11:15 on dist-test-slave-znh6"
server_key: "c972c39dbc94154b9e1fbf2abeb5b676"
server_key_iv: "b0aba0b661e57823db74978632a72ce1"
server_key_version: "encryptionkey@0"
I20260808 07:11:15.888936 10367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:15.889695 10390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:15.889868 10367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:15.889976 10367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "2731c7e0c2ae4728af3d1292738e7d5f"
format_stamp: "Formatted at 2026-08-08 07:11:15 on dist-test-slave-znh6"
server_key: "c972c39dbc94154b9e1fbf2abeb5b676"
server_key_iv: "b0aba0b661e57823db74978632a72ce1"
server_key_version: "encryptionkey@0"
I20260808 07:11:15.890049 10367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:15.896391 10367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:15.898121 10367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:15.898274 10367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:15.898485 10367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:15.898782 10367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:15.898859 10367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:15.898900 10367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:15.898929 10367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:15.905265 10367 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:41085
I20260808 07:11:15.905299 10561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:41085 every 8 connection(s)
I20260808 07:11:15.905609 10367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:15.909409 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 10367
I20260808 07:11:15.909541 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:15.909745 30937 external_mini_cluster.cc:1428] Setting key e358e9b796be3f61b4359500949f9c5c
Aug 08 07:11:15 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173075, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:11:15.915305 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.906248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:39105)
0808 07:11:15.906412 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:15.906445 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:15.906940 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:15.907144 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:15.907146 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:15.907741 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:15.907923 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.907927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.908737 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.908737 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.909534 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:15.909537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.909963 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.909964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:15.910109 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.910623 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:15.910633 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:15.912253 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:15.913849 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:15.913856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:15.913858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:15.914073 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:15.914296 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:15.914297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:15.914297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:15.914405 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:15.914786 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:15.914791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:15.914966 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 07:11:15.915050 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:11:15.915161 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260808 07:11:15.915355 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:15.906288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:39105 (local address 127.30.54.126:34041)
0808 07:11:15.906368 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:15.906371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:15.906509 (+ 138us) server_negotiation.cc:407] Connection header received
0808 07:11:15.907028 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:15.907029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:15.907042 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:15.907093 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:15.908000 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.908659 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.909616 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:15.909881 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:15.909943 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:15.912351 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:15.912352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:15.912353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:15.912361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:15.913747 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.914154 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.914155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.914155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.914217 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:15.914528 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:15.914530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:15.914530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:15.914706 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:15.914794 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:15.915106 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 07:11:15.915210 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:11:15.915807 10563 heartbeater.cc:347] Connected to a master server at 127.30.54.126:34041
I20260808 07:11:15.915894 10563 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:15.916126 10563 heartbeater.cc:511] Master 127.30.54.126:34041 requested a full tablet report, sending...
I20260808 07:11:15.916702 10259 ts_manager.cc:194] Registered new tserver with Master: 2731c7e0c2ae4728af3d1292738e7d5f (127.30.54.65:41085)
I20260808 07:11:15.917641 10259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:39105
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:11:15.938058 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34041
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:16.019363 10569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:16.019546 10569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:16.019569 10569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:16.020638 10569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:16.020673 10569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:16.020697 10569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:11:16.022025 10569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:16.022192 10569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:16.022365 10569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:16.027151 10578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.027172 10576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.027376 10569 server_base.cc:1034] running on GCE node
W20260808 07:11:16.027170 10575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.027694 10569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:16.027948 10569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:16.029137 10569 hybrid_clock.cc:648] HybridClock initialized: now 1786173076029124 us; error 30 us; skew 500 ppm
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:16.031747 10569 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:11:16.032163 10569 webserver.cc:460] Webserver started at http://127.30.54.66:43321/ using document root <none> and password file <none>
I20260808 07:11:16.032409 10569 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:16.032480 10569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:16.032647 10569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:16.033627 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "7fc564e75f21431c86361a769be5df2b"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-znh6"
server_key: "68aa81fb0f9904bf848a90e94ad56427"
server_key_iv: "efed998f9c61426cc7031aaeac1dfd95"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.033990 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "7fc564e75f21431c86361a769be5df2b"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-znh6"
server_key: "68aa81fb0f9904bf848a90e94ad56427"
server_key_iv: "efed998f9c61426cc7031aaeac1dfd95"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.035403 10569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:16.036146 10593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.036325 10569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:16.036423 10569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "7fc564e75f21431c86361a769be5df2b"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-znh6"
server_key: "68aa81fb0f9904bf848a90e94ad56427"
server_key_iv: "efed998f9c61426cc7031aaeac1dfd95"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.036504 10569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:16.047073 10569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:16.049206 10569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:16.049366 10569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:16.049587 10569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:16.049886 10569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:16.049940 10569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.049980 10569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:16.050009 10569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.056017 10569 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:45193
I20260808 07:11:16.056061 10768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:45193 every 8 connection(s)
I20260808 07:11:16.056375 10569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:11:16.063148 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 10569
I20260808 07:11:16.063279 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:16.063494 30937 external_mini_cluster.cc:1428] Setting key 4280abd125b32e95aea0bac360ff4e0d
I20260808 07:11:16.065575 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.057096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:49681 (local address 127.30.54.126:34041)
0808 07:11:16.057142 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.057144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.057352 (+ 208us) server_negotiation.cc:407] Connection header received
0808 07:11:16.057785 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.057786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.057797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.057855 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.058698 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.059205 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.060178 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.060358 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.060411 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.062460 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:16.062462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:16.062462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:16.062470 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:16.064037 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.064651 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.064652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.064652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.064702 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.064979 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.064980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.064980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.065074 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:16.065125 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:16.065446 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 07:11:16.065512 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:11:16.065910 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.057058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:49681)
0808 07:11:16.057212 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.057251 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.057706 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.057903 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.057906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.058443 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.058613 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.058617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.059309 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.059310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.060107 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.060109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.060432 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.060433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.060526 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.061019 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.061026 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.062345 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:16.064239 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:16.064246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:16.064247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:16.064572 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:16.064765 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:16.064766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:16.064766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:16.064912 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:16.065138 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:16.065143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:16.065307 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 07:11:16.065422 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:11:16.065589 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260808 07:11:16.066442 10769 heartbeater.cc:347] Connected to a master server at 127.30.54.126:34041
I20260808 07:11:16.066543 10769 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:16.066733 10769 heartbeater.cc:511] Master 127.30.54.126:34041 requested a full tablet report, sending...
I20260808 07:11:16.067117 10259 ts_manager.cc:194] Registered new tserver with Master: 7fc564e75f21431c86361a769be5df2b (127.30.54.66:45193)
I20260808 07:11:16.067498 10259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:49681
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:11:16.090330 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34041
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:16.172480 10774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:16.172705 10774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:16.172744 10774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:16.173878 10774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:16.173946 10774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:16.173991 10774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:11:16.175541 10774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:16.175791 10774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:16.176023 10774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:16.181206 10784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.181205 10786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.181305 10774 server_base.cc:1034] running on GCE node
W20260808 07:11:16.181200 10783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.181628 10774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:16.181854 10774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:16.183008 10774 hybrid_clock.cc:648] HybridClock initialized: now 1786173076182992 us; error 31 us; skew 500 ppm
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:16.185473 10774 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:11:16.185882 10774 webserver.cc:460] Webserver started at http://127.30.54.67:33581/ using document root <none> and password file <none>
I20260808 07:11:16.186122 10774 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:16.186167 10774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:16.186314 10774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:16.187363 10774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "7bcec0d0ceeb45f5bd70e58d461a6fec"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-znh6"
server_key: "5e32de30ca4904ba0aa222388b0d6011"
server_key_iv: "432b1b24651753a2d23a3e063f86d104"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.187717 10774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "7bcec0d0ceeb45f5bd70e58d461a6fec"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-znh6"
server_key: "5e32de30ca4904ba0aa222388b0d6011"
server_key_iv: "432b1b24651753a2d23a3e063f86d104"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.189007 10774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:16.189692 10798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.189863 10774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:16.189958 10774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "7bcec0d0ceeb45f5bd70e58d461a6fec"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-znh6"
server_key: "5e32de30ca4904ba0aa222388b0d6011"
server_key_iv: "432b1b24651753a2d23a3e063f86d104"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.190026 10774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:16.199779 10774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:16.201944 10774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:16.202105 10774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:16.202332 10774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:16.202636 10774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:16.202692 10774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.202723 10774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:16.202749 10774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.209174 10774 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:42725
I20260808 07:11:16.209206 10967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:42725 every 8 connection(s)
I20260808 07:11:16.209517 10774 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:16.215067 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 10774
I20260808 07:11:16.215199 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:16.215397 30937 external_mini_cluster.cc:1428] Setting key 7418f41ae0632e9020880812a1274a3b
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:11:16.218577 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.210176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:42133 (local address 127.30.54.126:34041)
0808 07:11:16.210215 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.210217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.210403 (+ 186us) server_negotiation.cc:407] Connection header received
0808 07:11:16.210924 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.210926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.210942 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.210992 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.211909 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.212418 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.213427 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.213641 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.213727 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.216004 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:16.216006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:16.216006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:16.216015 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:16.217491 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.217839 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.217840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.217840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.217891 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.218102 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.218103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.218103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.218199 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:16.218248 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:16.218459 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 07:11:16.218517 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:11:16.218748 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.210139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:42133)
0808 07:11:16.210283 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.210322 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.210783 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.211057 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.211061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.211629 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.211804 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.211808 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.212525 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.212526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.213335 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.213337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.213737 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.213739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.213859 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.214239 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.214246 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.215901 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:16.217565 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:16.217570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:16.217572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:16.217774 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:16.217957 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:16.217957 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:16.217958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:16.218049 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:16.218251 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:16.218255 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:16.218352 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 07:11:16.218444 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:11:16.218558 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260808 07:11:16.219223 10969 heartbeater.cc:347] Connected to a master server at 127.30.54.126:34041
I20260808 07:11:16.219309 10969 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:16.219439 10969 heartbeater.cc:511] Master 127.30.54.126:34041 requested a full tablet report, sending...
I20260808 07:11:16.219764 10259 ts_manager.cc:194] Registered new tserver with Master: 7bcec0d0ceeb45f5bd70e58d461a6fec (127.30.54.67:42725)
I20260808 07:11:16.220211 10259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:42133
I20260808 07:11:16.227512 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab.
Entry for principal oryx/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab.
WARNING: no policy specified for oryx/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab.
Entry for principal oryx/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab.
WARNING: no policy specified for oryx/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab.
Entry for principal oryx/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab.
I20260808 07:11:16.297979 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 10367
I20260808 07:11:16.306051 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 10569
I20260808 07:11:16.311901 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 10774
I20260808 07:11:16.322808 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 10215
I20260808 07:11:16.337059 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34041
--webserver_interface=127.30.54.126
--webserver_port=37703
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:34041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:16.420293 10981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:16.420496 10981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:16.420519 10981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:16.421713 10981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:16.421754 10981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:16.421769 10981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:16.421783 10981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:16.421795 10981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:16.423307 10981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:34041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=37703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:16.423545 10981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:16.423780 10981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:16.428625 10981 server_base.cc:1034] running on GCE node
W20260808 07:11:16.428597 10994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.428592 10990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.428582 10987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.429029 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:16.429359 10981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:16.430523 10981 hybrid_clock.cc:648] HybridClock initialized: now 1786173076430520 us; error 37 us; skew 500 ppm
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:16.433050 10981 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:11:16.433419 10981 webserver.cc:460] Webserver started at http://127.30.54.126:37703/ using document root <none> and password file <none>
I20260808 07:11:16.433590 10981 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:16.433643 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:16.434974 10981 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:16.435624 11004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.435796 10981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:16.435882 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "9943289736554a7dab7ae6debb82b62d"
format_stamp: "Formatted at 2026-08-08 07:11:15 on dist-test-slave-znh6"
server_key: "6191b732b2f404bbcdb69ba325e90fae"
server_key_iv: "51b5226bcce66f9313bf6e56673e8236"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.436174 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:16.451330 10981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:16.453470 10981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:16.453576 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:16.457520 10981 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:34041
I20260808 07:11:16.457593 11084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:34041 every 8 connection(s)
I20260808 07:11:16.457953 10981 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:16.459383 11085 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:11:16.460049 11085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: Bootstrap starting.
I20260808 07:11:16.461887 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 10981
I20260808 07:11:16.462199 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:41085
--local_ip_for_outbound_sockets=127.30.54.65
--tserver_master_addrs=127.30.54.126:34041
--webserver_port=46275
--webserver_interface=127.30.54.65
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.54.65
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:16.468911 11085 log.cc:824] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:16.470489 11085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:11:16.470770 11085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: Bootstrap complete.
I20260808 07:11:16.472972 11085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } }
I20260808 07:11:16.473215 11085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9943289736554a7dab7ae6debb82b62d, State: Initialized, Role: FOLLOWER
I20260808 07:11:16.473394 11085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } }
I20260808 07:11:16.473476 11085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:16.473528 11085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:16.473580 11085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:11:16.474553 11085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } }
I20260808 07:11:16.474677 11085 leader_election.cc:302] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9943289736554a7dab7ae6debb82b62d; no voters:
I20260808 07:11:16.474882 11085 leader_election.cc:288] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:11:16.474918 11090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:11:16.475044 11090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [term 2 LEADER]: Becoming Leader. State: Replica: 9943289736554a7dab7ae6debb82b62d, State: Running, Role: LEADER
I20260808 07:11:16.475165 11085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:16.475198 11090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } }
I20260808 07:11:16.475615 11092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9943289736554a7dab7ae6debb82b62d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } } }
I20260808 07:11:16.475711 11092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:16.476030 11099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:16.476135 11093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9943289736554a7dab7ae6debb82b62d. Latest consensus state: current_term: 2 leader_uuid: "9943289736554a7dab7ae6debb82b62d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9943289736554a7dab7ae6debb82b62d" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34041 } } }
I20260808 07:11:16.476285 11093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:16.476573 11099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:16.476696 11099 catalog_manager.cc:1259] Loaded cluster ID: d7a3f817228449a09c88d4f838f96ca5
I20260808 07:11:16.476771 11099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:16.478119 11099 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:16.478281 11099 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9943289736554a7dab7ae6debb82b62d: Loaded TSK: 0
I20260808 07:11:16.478509 11099 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:11:16.545378 11087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:16.545570 11087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:16.545593 11087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:16.546713 11087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:16.546753 11087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:16.546779 11087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:11:16.548206 11087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.65.keytab
--principal=oryx/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:41085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=46275
--tserver_master_addrs=127.30.54.126:34041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:16.548426 11087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:16.548663 11087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:16.553699 11131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.553673 11130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.553952 11133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.554124 11087 server_base.cc:1034] running on GCE node
I20260808 07:11:16.554324 11087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:16.554555 11087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:16.555712 11087 hybrid_clock.cc:648] HybridClock initialized: now 1786173076555690 us; error 42 us; skew 500 ppm
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:16.558136 11087 init.cc:377] Logged in from keytab as oryx/127.30.54.65@KRBTEST.COM (short username oryx)
I20260808 07:11:16.558583 11087 webserver.cc:460] Webserver started at http://127.30.54.65:46275/ using document root <none> and password file <none>
I20260808 07:11:16.558847 11087 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:16.558952 11087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:16.560282 11087 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:16.560931 11145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.561105 11087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:16.561203 11087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "2731c7e0c2ae4728af3d1292738e7d5f"
format_stamp: "Formatted at 2026-08-08 07:11:15 on dist-test-slave-znh6"
server_key: "c972c39dbc94154b9e1fbf2abeb5b676"
server_key_iv: "b0aba0b661e57823db74978632a72ce1"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.561537 11087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:16.579660 11087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:16.582037 11087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:16.582206 11087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:16.582454 11087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:16.582777 11087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:16.582858 11087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.582903 11087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:16.582929 11087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.589358 11087 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:41085
I20260808 07:11:16.589427 11326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:41085 every 8 connection(s)
I20260808 07:11:16.589787 11087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:16.598453 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 11087
I20260808 07:11:16.598762 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:45193
--local_ip_for_outbound_sockets=127.30.54.66
--tserver_master_addrs=127.30.54.126:34041
--webserver_port=43321
--webserver_interface=127.30.54.66
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.54.66
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:16.599005 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.590537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:39769)
0808 07:11:16.590715 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.590755 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.591263 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.591775 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.591778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.592318 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.592497 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.592502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.594248 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.594249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.595111 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.595113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.595883 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.595884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.595986 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.596494 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.596502 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.598767 (+ 2265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
W20260808 07:11:16.599048 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:16.590620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:39769 (local address 127.30.54.126:34041)
0808 07:11:16.590805 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.590833 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.590869 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:11:16.591492 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.591502 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.591568 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.591739 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.592625 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.594119 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.595199 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.595789 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.595896 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.598907 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:39769: BlockingRecv error: recv got EOF from 127.30.54.65:39769 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
W20260808 07:11:16.599274 11328 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:34041 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.54.126:34041: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:16.604449 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.599498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:57093)
0808 07:11:16.599631 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.599640 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.599672 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.599959 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.599959 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.600084 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.600124 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.600125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.601027 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.601028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.601803 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.601806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.602165 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.602166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.602229 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.602624 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.602629 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.604366 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120}
W20260808 07:11:16.604502 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:16.599596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:57093 (local address 127.30.54.126:34041)
0808 07:11:16.599847 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.599849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.599856 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:11:16.599873 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.599874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.599883 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.599955 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.600201 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.600918 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.601895 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.602092 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.602167 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.604438 (+ 2271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:57093: BlockingRecv error: recv got EOF from 127.30.54.65:57093 (error 108)
Metrics: {"server-negotiator.queue_time_us":233}
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:11:16.609556 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:16.605003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:42581 (local address 127.30.54.126:34041)
0808 07:11:16.605061 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.605063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.605116 (+ 53us) server_negotiation.cc:407] Connection header received
0808 07:11:16.605194 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.605194 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.605202 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.605245 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.605552 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.605971 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.607049 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.607216 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.607266 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.609492 (+ 2226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:42581: BlockingRecv error: recv got EOF from 127.30.54.65:42581 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:11:16.609503 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.604920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:42581)
0808 07:11:16.605059 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.605072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.605134 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.605279 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.605280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.605440 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.605477 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.605479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.606050 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.606052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.606967 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.606970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.607292 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.607293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.607358 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.607690 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.607695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.609429 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119}
W20260808 07:11:16.609717 11328 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:11:16.682140 11332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:16.682335 11332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:16.682358 11332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:16.683542 11332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:16.683584 11332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:16.683610 11332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:11:16.685005 11332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.66.keytab
--principal=oryx/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:45193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=43321
--tserver_master_addrs=127.30.54.126:34041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:16.685235 11332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:16.685462 11332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:16.690442 11348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.690542 11345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.690624 11332 server_base.cc:1034] running on GCE node
W20260808 07:11:16.690439 11339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.690982 11332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:16.691249 11332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:16.692437 11332 hybrid_clock.cc:648] HybridClock initialized: now 1786173076692423 us; error 27 us; skew 500 ppm
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:16.695220 11332 init.cc:377] Logged in from keytab as oryx/127.30.54.66@KRBTEST.COM (short username oryx)
I20260808 07:11:16.695694 11332 webserver.cc:460] Webserver started at http://127.30.54.66:43321/ using document root <none> and password file <none>
I20260808 07:11:16.695931 11332 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:16.696023 11332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:16.697422 11332 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:16.698055 11355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.698242 11332 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:16.698340 11332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "7fc564e75f21431c86361a769be5df2b"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-znh6"
server_key: "68aa81fb0f9904bf848a90e94ad56427"
server_key_iv: "efed998f9c61426cc7031aaeac1dfd95"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.698657 11332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:16.708729 11332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:16.710947 11332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:16.711122 11332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:16.711334 11332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:16.711642 11332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:16.711695 11332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.711737 11332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:16.711764 11332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.717460 11332 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:45193
I20260808 07:11:16.717499 11525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:45193 every 8 connection(s)
I20260808 07:11:16.717833 11332 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:16.724470 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.718476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:54343)
0808 07:11:16.718637 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.718676 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.719176 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.719420 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.719423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.719944 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.720113 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.720116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.720746 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.720747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.721515 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.721517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.721838 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.721839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.721931 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.722315 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.722323 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.724342 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
W20260808 07:11:16.724517 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:16.718516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:54343 (local address 127.30.54.126:34041)
0808 07:11:16.718562 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.718564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.718772 (+ 208us) server_negotiation.cc:407] Connection header received
0808 07:11:16.719264 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.719265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.719276 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.719333 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.720191 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.720671 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.721592 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.721769 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.721817 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.724459 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:54343: BlockingRecv error: recv got EOF from 127.30.54.66:54343 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:11:16.724861 11526 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:34041 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.54.126:34041: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
I20260808 07:11:16.725067 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 11332
I20260808 07:11:16.725373 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:42725
--local_ip_for_outbound_sockets=127.30.54.67
--tserver_master_addrs=127.30.54.126:34041
--webserver_port=33581
--webserver_interface=127.30.54.67
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.54.67
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:16.729519 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.725096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:43389)
0808 07:11:16.725156 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.725164 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.725215 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.725367 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.725368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.725543 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.725603 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.725604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.726435 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.726436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.727004 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.727006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.727397 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.727398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.727466 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.727868 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.727872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.729480 (+ 1608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 07:11:16.729663 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:16.725153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:43389 (local address 127.30.54.126:34041)
0808 07:11:16.725220 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.725222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.725251 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:11:16.725271 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.725271 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.725294 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.725349 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.725678 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.726348 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.727089 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.727309 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.727395 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.729587 (+ 2192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:43389: BlockingRecv error: recv got EOF from 127.30.54.66:43389 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:16.734226 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.729990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:38767)
0808 07:11:16.730039 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.730051 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.730095 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.730251 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.730251 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.730352 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.730375 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.730376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.731314 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.731314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.731774 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.731775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.732257 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.732258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.732292 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.732666 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.732670 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.734189 (+ 1519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:11:16.734349 11526 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:11:16.734380 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:16.730072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:38767 (local address 127.30.54.126:34041)
0808 07:11:16.730135 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.730138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.730148 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:16.730170 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.730171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.730181 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.730250 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.730445 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.731237 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.731854 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.732175 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.732245 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.734299 (+ 2054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:38767: BlockingRecv error: recv got EOF from 127.30.54.66:38767 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 07:11:16.807766 11531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:16.807956 11531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:16.807979 11531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:16.809056 11531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:16.809092 11531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:16.809118 11531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:11:16.810549 11531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/krb5kdc/oryx_127.30.54.67.keytab
--principal=oryx/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:42725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=33581
--tserver_master_addrs=127.30.54.126:34041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:16.810731 11531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:16.810905 11531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:16.815810 11542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.815796 11538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:16.815778 11539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:16.816052 11531 server_base.cc:1034] running on GCE node
I20260808 07:11:16.816244 11531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:16.816478 11531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:16.817636 11531 hybrid_clock.cc:648] HybridClock initialized: now 1786173076817638 us; error 43 us; skew 500 ppm
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173076, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:16.820025 11531 init.cc:377] Logged in from keytab as oryx/127.30.54.67@KRBTEST.COM (short username oryx)
I20260808 07:11:16.820500 11531 webserver.cc:460] Webserver started at http://127.30.54.67:33581/ using document root <none> and password file <none>
I20260808 07:11:16.820735 11531 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:16.820820 11531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:16.822175 11531 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:16.822875 11556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.823093 11531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:16.823207 11531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "7bcec0d0ceeb45f5bd70e58d461a6fec"
format_stamp: "Formatted at 2026-08-08 07:11:16 on dist-test-slave-znh6"
server_key: "5e32de30ca4904ba0aa222388b0d6011"
server_key_iv: "432b1b24651753a2d23a3e063f86d104"
server_key_version: "encryptionkey@0"
I20260808 07:11:16.823573 11531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:16.832988 11531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:16.835217 11531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:16.835379 11531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:16.835594 11531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:16.835884 11531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:16.835937 11531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.835976 11531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:16.836004 11531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:16.842614 11531 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:42725
I20260808 07:11:16.842648 11738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:42725 every 8 connection(s)
I20260808 07:11:16.843026 11531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:16.849746 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.843729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:57249)
0808 07:11:16.843882 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.843915 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.844372 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.844567 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.844570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.845106 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.845292 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.845296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.845914 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.845915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.846686 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.846688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.847072 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.847073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.847168 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.847560 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.847567 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.849620 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
W20260808 07:11:16.849758 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:16.843738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:57249 (local address 127.30.54.126:34041)
0808 07:11:16.843810 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.843812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.843995 (+ 183us) server_negotiation.cc:407] Connection header received
0808 07:11:16.844455 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.844456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.844467 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.844519 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.845365 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.845840 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.846759 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.847000 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.847054 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.849712 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:57249: BlockingRecv error: recv got EOF from 127.30.54.67:57249 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 07:11:16.850106 11739 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:34041 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.54.126:34041: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
I20260808 07:11:16.851223 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 11531
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:16.855454 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.850359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:42503)
0808 07:11:16.850406 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.850416 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.850453 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.850656 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.850657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.850761 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.850806 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.850807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.851686 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.851687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.852540 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.852543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.853048 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.853049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.853099 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.853502 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.853506 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.855399 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:11:16.855566 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:16.850501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:42503 (local address 127.30.54.126:34041)
0808 07:11:16.850556 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.850558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.850566 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:16.850591 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.850592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.850598 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.850643 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.850908 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.851581 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.852657 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.852967 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.853043 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.855502 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:42503: BlockingRecv error: recv got EOF from 127.30.54.67:42503 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:16.859500 11744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.851657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.30.54.126:34041)
0808 07:11:16.851796 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.851803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.851876 (+ 73us) server_negotiation.cc:407] Connection header received
0808 07:11:16.851943 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.851943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.851952 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.852007 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.852711 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.853298 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.854222 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.854614 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.854694 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.856126 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:16.856130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:16.856130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:16.856141 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:16.857754 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.858389 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.858392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.858393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.858499 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:16.858709 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:16.858710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:16.858710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:16.859030 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:16.859170 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:16.859284 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 07:11:16.859396 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
Aug 08 07:11:16 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:16.860320 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:16.855975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:57305)
0808 07:11:16.856028 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:16.856039 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:16.856085 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:16.856212 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:16.856213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:16.856375 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:16.856405 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.856407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.857290 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.857290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.857739 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:16.857741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.858107 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.858107 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:16.858144 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.858538 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:16.858543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:16.860249 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 07:11:16.860373 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:16.856051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:57305 (local address 127.30.54.126:34041)
0808 07:11:16.856089 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:16.856090 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:16.856098 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:16.856144 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:16.856145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:16.856153 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:16.856214 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:16.856483 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.857207 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.857832 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:16.858044 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:16.858093 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:16.860313 (+ 2220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:57305: BlockingRecv error: recv got EOF from 127.30.54.67:57305 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:11:16.860589 11739 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 07:11:17 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:17 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:17.614951 11746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.610098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:35133)
0808 07:11:17.610273 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.610285 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.610338 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.610505 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.610506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.610669 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:17.610720 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.610722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.611413 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.611414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.612001 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.612004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.612493 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.612494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.612567 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.613045 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:17.613050 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:17.614884 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
W20260808 07:11:17.615067 11747 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:17.610193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:35133 (local address 127.30.54.126:34041)
0808 07:11:17.610326 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.610328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.610338 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:17.610411 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.610411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.610420 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.610491 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:17.610785 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.611333 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.612123 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.612396 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.612482 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.614970 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:35133: BlockingRecv error: recv got EOF from 127.30.54.65:35133 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
Aug 08 07:11:17 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:17 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:17.739428 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.734714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:46411)
0808 07:11:17.734892 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.734902 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.734968 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.735077 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.735078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.735235 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:17.735286 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.735287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.735918 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.735920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.736503 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.736505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.736827 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.736828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.736869 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.737394 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:17.737399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:17.739353 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
W20260808 07:11:17.739528 11747 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:17.734801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:46411 (local address 127.30.54.126:34041)
0808 07:11:17.734909 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.734912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.734967 (+ 55us) server_negotiation.cc:407] Connection header received
0808 07:11:17.735010 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.735011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.735022 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.735075 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:17.735340 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.735835 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.736573 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.736755 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.736805 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.739462 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:46411: BlockingRecv error: recv got EOF from 127.30.54.66:46411 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 07:11:17 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:17 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:17.865854 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:17.860971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:53069)
0808 07:11:17.861142 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:17.861153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:17.861221 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:17.861358 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:17.861359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:17.861543 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:17.861593 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.861595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.862278 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.862279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.862909 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:17.862911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.863311 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.863312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:17.863370 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.863874 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:17.863882 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:17.865749 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
W20260808 07:11:17.865909 11747 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:17.861053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:53069 (local address 127.30.54.126:34041)
0808 07:11:17.861123 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:17.861125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:17.861201 (+ 76us) server_negotiation.cc:407] Connection header received
0808 07:11:17.861274 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:17.861275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:17.861289 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:17.861342 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:17.861677 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.862190 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.862997 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:17.863208 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:17.863294 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:17.865840 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:53069: BlockingRecv error: recv got EOF from 127.30.54.67:53069 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 07:11:18 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:18 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:18.619956 11752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.615410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:43639)
0808 07:11:18.615575 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:18.615589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:18.615657 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:18.615807 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:18.615808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:18.615968 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:18.616020 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.616021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.616653 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.616653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.617188 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.617189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.617529 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.617529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.617573 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.618011 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:18.618015 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:18.619900 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
W20260808 07:11:18.620066 11754 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:18.615466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:43639 (local address 127.30.54.126:34041)
0808 07:11:18.615629 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:18.615630 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:18.615643 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:11:18.615712 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:18.615712 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:18.615723 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:18.615782 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:18.616080 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.616586 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.617264 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.617462 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.617515 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.619988 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:43639: BlockingRecv error: recv got EOF from 127.30.54.65:43639 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
Aug 08 07:11:18 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:18 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:18.744732 11756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.739916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:39663)
0808 07:11:18.740161 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:18.740173 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:18.740260 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:18.740462 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:18.740463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:18.740692 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:18.740754 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.740757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.741379 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.741379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.741910 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.741912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.742248 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.742249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.742288 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.742717 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:18.742722 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:18.744677 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":67,"threads_started":1}
W20260808 07:11:18.744875 11754 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:18.740015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:39663 (local address 127.30.54.126:34041)
0808 07:11:18.740075 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:18.740077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:18.740321 (+ 244us) server_negotiation.cc:407] Connection header received
0808 07:11:18.740351 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:18.740351 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:18.740363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:18.740432 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:18.740822 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.741314 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.741986 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.742177 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.742236 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.744776 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:39663: BlockingRecv error: recv got EOF from 127.30.54.66:39663 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 07:11:18 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:18 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:18.871233 11759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:18.866374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:60493)
0808 07:11:18.866555 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:18.866568 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:18.866640 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:18.866791 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:18.866792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:18.867003 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:18.867056 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.867058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.867754 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.867755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.868306 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:18.868315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.868686 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.868687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:18.868755 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.869247 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:18.869251 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:18.871134 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
W20260808 07:11:18.871284 11754 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:18.866514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:60493 (local address 127.30.54.126:34041)
0808 07:11:18.866562 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:18.866564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:18.866637 (+ 73us) server_negotiation.cc:407] Connection header received
0808 07:11:18.866698 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:18.866699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:18.866714 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:18.866785 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:18.867145 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.867664 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.868406 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:18.868605 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:18.868687 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:18.871217 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:60493: BlockingRecv error: recv got EOF from 127.30.54.67:60493 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 07:11:19 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:19 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:19.625080 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:19.620460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:43003)
0808 07:11:19.620627 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:19.620641 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:19.620713 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:19.620853 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:19.620854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:19.621013 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:19.621066 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:19.621067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.621707 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.621708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:19.622217 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:19.622219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.622548 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.622549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:19.622593 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:19.623149 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:19.623154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:19.625023 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
W20260808 07:11:19.625190 11763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:19.620565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:43003 (local address 127.30.54.126:34041)
0808 07:11:19.620721 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:19.620722 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:19.620732 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:19.620755 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:19.620756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:19.620766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:19.620830 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:19.621130 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.621641 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.622288 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.622474 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.622529 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:19.625115 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:43003: BlockingRecv error: recv got EOF from 127.30.54.65:43003 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
Aug 08 07:11:19 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:19 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:19.750507 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:19.745249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:45349)
0808 07:11:19.745434 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:19.745448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:19.745554 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:19.745712 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:19.745713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:19.745914 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:19.745964 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:19.745966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.746668 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.746669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:19.747234 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:19.747235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.747641 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.747642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:19.747704 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:19.748211 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:19.748217 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:19.750450 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
W20260808 07:11:19.750623 11763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:19.745356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:45349 (local address 127.30.54.126:34041)
0808 07:11:19.745439 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:19.745442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:19.745532 (+ 90us) server_negotiation.cc:407] Connection header received
0808 07:11:19.745614 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:19.745615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:19.745628 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:19.745704 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:19.746061 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.746573 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.747327 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.747533 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.747640 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:19.750548 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:45349: BlockingRecv error: recv got EOF from 127.30.54.66:45349 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 07:11:19 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:19 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:19.876693 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:19.871722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:49401)
0808 07:11:19.871919 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:19.871932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:19.872025 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:19.872155 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:19.872156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:19.872345 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:19.872394 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:19.872396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.873094 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.873095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:19.873628 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:19.873630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.874006 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.874007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:19.874067 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:19.874572 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:19.874582 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:19.876597 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":104,"threads_started":1}
W20260808 07:11:19.876742 11763 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:19.871830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:49401 (local address 127.30.54.126:34041)
0808 07:11:19.871916 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:19.871919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:19.872004 (+ 85us) server_negotiation.cc:407] Connection header received
0808 07:11:19.872079 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:19.872080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:19.872096 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:19.872149 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:19.872484 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.873005 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.873728 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:19.873931 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:19.874007 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:19.876675 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:49401: BlockingRecv error: recv got EOF from 127.30.54.67:49401 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 07:11:20 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:20 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:20.630421 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.625609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:53193)
0808 07:11:20.625750 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:20.625762 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:20.625832 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:20.626040 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:20.626042 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:20.626276 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:20.626341 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.626343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.627156 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.627157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.627680 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.627681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.628008 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.628009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.628051 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.628507 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:20.628512 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:20.630316 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260808 07:11:20.630466 11768 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:20.625695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:53193 (local address 127.30.54.126:34041)
0808 07:11:20.625825 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:20.625827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:20.625835 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:20.625908 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:20.625908 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:20.625918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:20.626012 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:20.626424 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.627086 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.627751 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.627945 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.627994 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.630395 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:53193: BlockingRecv error: recv got EOF from 127.30.54.65:53193 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
Aug 08 07:11:20 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:20 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:20.758410 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.750997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:57265)
0808 07:11:20.751148 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:20.751158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:20.751216 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:20.751357 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:20.751358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:20.751532 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:20.751584 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.751585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.752212 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.752213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.752741 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.752742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.753081 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.753082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.753122 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.756670 (+ 3548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:20.756675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:20.758354 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260808 07:11:20.758517 11768 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:20.751080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:57265 (local address 127.30.54.126:34041)
0808 07:11:20.751151 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:20.751153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:20.751217 (+ 64us) server_negotiation.cc:407] Connection header received
0808 07:11:20.751259 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:20.751260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:20.751270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:20.751323 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:20.751648 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.752143 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.752814 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.752996 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.753084 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.758452 (+ 5368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:57265: BlockingRecv error: recv got EOF from 127.30.54.66:57265 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 07:11:20 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:20 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:20.881882 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:20.877162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:55655)
0808 07:11:20.877372 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:20.877383 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:20.877449 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:20.877603 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:20.877604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:20.877795 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:20.877843 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.877844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.878523 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.878524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.879098 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:20.879100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.879484 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.879485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:20.879545 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.880048 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:20.880052 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:20.881823 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":123,"threads_started":1}
W20260808 07:11:20.882004 11768 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:20.877247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:55655 (local address 127.30.54.126:34041)
0808 07:11:20.877353 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:20.877355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:20.877441 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:11:20.877506 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:20.877507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:20.877521 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:20.877596 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:20.877925 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.878437 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.879197 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:20.879390 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:20.879482 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:20.881926 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:55655: BlockingRecv error: recv got EOF from 127.30.54.67:55655 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 07:11:21 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:21 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:21.635536 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:21.630993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:53689)
0808 07:11:21.631144 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:21.631155 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:21.631206 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:21.631337 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:21.631337 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:21.631527 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:21.631577 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:21.631579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.632229 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.632230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:21.632754 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:21.632756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.633096 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.633097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:21.633143 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:21.633642 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:21.633647 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:21.635479 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
W20260808 07:11:21.635665 11772 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:21.631071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:53689 (local address 127.30.54.126:34041)
0808 07:11:21.631195 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:21.631201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:21.631210 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:21.631250 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:21.631250 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:21.631263 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:21.631324 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:21.631648 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.632163 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.632826 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.633024 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.633080 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:21.635575 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:53689: BlockingRecv error: recv got EOF from 127.30.54.65:53689 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
Aug 08 07:11:21 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:21 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:21.763622 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:21.758986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:32855)
0808 07:11:21.759133 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:21.759145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:21.759206 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:21.759324 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:21.759324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:21.759508 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:21.759559 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:21.759561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.760231 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.760233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:21.760838 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:21.760841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.761184 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.761184 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:21.761235 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:21.761727 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:21.761732 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:21.763565 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":60,"threads_started":1}
W20260808 07:11:21.763763 11772 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:21.759139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:32855 (local address 127.30.54.126:34041)
0808 07:11:21.759207 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:21.759209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:21.759217 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:21.759245 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:21.759246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:21.759256 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:21.759305 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:21.759632 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.760140 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.760918 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.761107 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.761157 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:21.763676 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:32855: BlockingRecv error: recv got EOF from 127.30.54.66:32855 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 07:11:21 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:21 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:21.887341 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:21.882398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:34665)
0808 07:11:21.882548 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:21.882558 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:21.882614 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:21.882733 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:21.882734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:21.882914 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:21.882967 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:21.882969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.883653 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.883654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:21.884437 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:21.884440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.884824 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.884825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:21.884868 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:21.885377 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:21.885383 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:21.887233 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
W20260808 07:11:21.887388 11772 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:21.882492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:34665 (local address 127.30.54.126:34041)
0808 07:11:21.882537 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:21.882539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:21.882603 (+ 64us) server_negotiation.cc:407] Connection header received
0808 07:11:21.882675 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:21.882676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:21.882686 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:21.882727 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:21.883029 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.883552 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.884530 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:21.884758 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:21.884813 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:21.887323 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:34665: BlockingRecv error: recv got EOF from 127.30.54.67:34665 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 07:11:22 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:22 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:22.641160 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:22.636081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:37155)
0808 07:11:22.636314 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:22.636328 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:22.636404 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:22.636576 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:22.636577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:22.636768 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:22.636820 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:22.636822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.637468 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.637470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:22.638007 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:22.638009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.638423 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.638425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:22.638479 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:22.639095 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:22.639100 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:22.641092 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":76,"threads_started":1}
W20260808 07:11:22.641291 11777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:22.636137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:37155 (local address 127.30.54.126:34041)
0808 07:11:22.636263 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:22.636265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:22.636399 (+ 134us) server_negotiation.cc:407] Connection header received
0808 07:11:22.636468 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:22.636469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:22.636483 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:22.636559 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:22.636887 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.637390 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.638079 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.638340 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.638392 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:22.641178 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:37155: BlockingRecv error: recv got EOF from 127.30.54.65:37155 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
Aug 08 07:11:22 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:22 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:22.768836 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:22.764078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:45157)
0808 07:11:22.764240 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:22.764273 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:22.764349 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:22.764482 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:22.764483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:22.764641 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:22.764692 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:22.764694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.765429 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.765430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:22.766162 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:22.766164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.766492 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.766493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:22.766534 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:22.766982 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:22.766987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:22.768779 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":109,"threads_started":1}
W20260808 07:11:22.768939 11777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:22.764130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:45157 (local address 127.30.54.126:34041)
0808 07:11:22.764207 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:22.764208 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:22.764324 (+ 116us) server_negotiation.cc:407] Connection header received
0808 07:11:22.764398 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:22.764398 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:22.764409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:22.764462 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:22.764761 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.765361 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.766242 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.766428 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.766483 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:22.768871 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:45157: BlockingRecv error: recv got EOF from 127.30.54.66:45157 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 07:11:22 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:22 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:22.892421 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:22.887897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:45951)
0808 07:11:22.888056 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:22.888068 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:22.888134 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:22.888263 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:22.888264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:22.888446 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:22.888499 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:22.888501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.889158 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.889160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:22.889709 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:22.889710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.890027 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.890028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:22.890069 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:22.890506 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:22.890510 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:22.892364 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
W20260808 07:11:22.892524 11777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:22.887952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:45951 (local address 127.30.54.126:34041)
0808 07:11:22.888030 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:22.888031 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:22.888135 (+ 104us) server_negotiation.cc:407] Connection header received
0808 07:11:22.888177 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:22.888178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:22.888190 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:22.888243 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:22.888562 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.889081 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.889782 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:22.889956 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:22.890012 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:22.892454 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:45951: BlockingRecv error: recv got EOF from 127.30.54.67:45951 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 07:11:23 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:23 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:23.646095 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.641655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:60051)
0808 07:11:23.641815 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:23.641826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:23.641877 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:23.642023 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:23.642024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:23.642188 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:23.642240 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.642242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.642960 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.642960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.643501 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.643502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.643832 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.643833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.643883 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.644351 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:23.644355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:23.646037 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
W20260808 07:11:23.646186 11784 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:23.641714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:60051 (local address 127.30.54.126:34041)
0808 07:11:23.641862 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:23.641864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:23.641873 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:23.641922 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:23.641923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:23.641934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:23.642001 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:23.642310 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.642886 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.643574 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.643766 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.643816 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.646132 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:60051: BlockingRecv error: recv got EOF from 127.30.54.65:60051 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
Aug 08 07:11:23 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:23 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:23.774487 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.769345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:34177)
0808 07:11:23.769560 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:23.769572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:23.769672 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:23.769830 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:23.769831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:23.770020 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:23.770068 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.770070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.770750 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.770751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.771309 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.771311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.771718 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.771719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.771783 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.772326 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:23.772332 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:23.774426 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":126,"threads_started":1}
W20260808 07:11:23.774621 11784 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:23.769453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:34177 (local address 127.30.54.126:34041)
0808 07:11:23.769554 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:23.769557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:23.769653 (+ 96us) server_negotiation.cc:407] Connection header received
0808 07:11:23.769734 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:23.769735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:23.769754 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:23.769824 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:23.770152 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.770664 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.771400 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.771610 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.771717 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.774537 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:34177: BlockingRecv error: recv got EOF from 127.30.54.66:34177 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 07:11:23 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:23 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:23.897680 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:23.892963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:36795)
0808 07:11:23.893119 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:23.893130 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:23.893207 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:23.893360 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:23.893361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:23.893531 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:23.893584 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.893585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.894301 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.894302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.894910 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:23.894912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.895291 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.895292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:23.895345 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.895904 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:23.895909 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:23.897623 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
W20260808 07:11:23.897780 11784 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:23.893059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:36795 (local address 127.30.54.126:34041)
0808 07:11:23.893130 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:23.893132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:23.893215 (+ 83us) server_negotiation.cc:407] Connection header received
0808 07:11:23.893254 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:23.893254 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:23.893267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:23.893345 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:23.893652 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.894221 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.895002 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:23.895199 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:23.895265 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:23.897713 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:36795: BlockingRecv error: recv got EOF from 127.30.54.67:36795 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 07:11:24 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:24 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:24.652390 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:24.646580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:35689)
0808 07:11:24.646724 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:24.646738 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:24.646835 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:24.647050 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:24.647051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:24.647241 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:24.647303 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:24.647307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.648420 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.648421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:24.649026 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:24.649030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.649479 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.649481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:24.649551 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:24.650081 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:24.650087 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:24.652281 (+ 2194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
W20260808 07:11:24.652434 11789 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:24.646681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:35689 (local address 127.30.54.126:34041)
0808 07:11:24.646891 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:24.646894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:24.646905 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:24.646936 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:24.646937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:24.646949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:24.647034 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:24.647385 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.648308 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.649121 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.649383 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.649452 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:24.652360 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:35689: BlockingRecv error: recv got EOF from 127.30.54.65:35689 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":105,"threads_started":1}
Aug 08 07:11:24 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:24 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:24.780612 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:24.775025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:55933)
0808 07:11:24.775185 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:24.775200 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:24.775267 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:24.775455 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:24.775457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:24.775713 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:24.775774 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:24.775777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.776775 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.776776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:24.777308 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:24.777310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.777821 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.777822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:24.777895 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:24.778338 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:24.778342 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:24.780544 (+ 2202us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":63,"threads_started":1}
W20260808 07:11:24.780732 11789 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:24.775202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:55933 (local address 127.30.54.126:34041)
0808 07:11:24.775268 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:24.775269 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:24.775278 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:24.775320 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:24.775321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:24.775330 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:24.775398 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:24.775863 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.776698 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.777407 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.777740 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.777812 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:24.780637 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:55933: BlockingRecv error: recv got EOF from 127.30.54.66:55933 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 07:11:24 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:24 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:24.903187 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:24.898179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:36581)
0808 07:11:24.898388 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:24.898407 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:24.898486 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:24.898642 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:24.898643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:24.898883 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:24.898944 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:24.898946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.899619 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.899621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:24.900163 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:24.900164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.900525 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.900526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:24.900586 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:24.901066 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:24.901072 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:24.903073 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":112,"threads_started":1}
W20260808 07:11:24.903226 11789 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:24.898279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:36581 (local address 127.30.54.126:34041)
0808 07:11:24.898356 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:24.898358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:24.898470 (+ 112us) server_negotiation.cc:407] Connection header received
0808 07:11:24.898543 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:24.898544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:24.898557 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:24.898634 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:24.899030 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.899535 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.900247 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:24.900443 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:24.900514 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:24.903160 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:36581: BlockingRecv error: recv got EOF from 127.30.54.67:36581 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 07:11:25 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:25 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:25.658049 11793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:25.652994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:53401)
0808 07:11:25.653217 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:25.653230 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:25.653337 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:25.653561 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:25.653562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:25.653812 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:25.653875 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:25.653878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.654598 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.654599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:25.655207 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:25.655209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.655573 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.655574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:25.655630 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:25.656078 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:25.656082 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:25.657991 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":93,"threads_started":1}
W20260808 07:11:25.658208 11794 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:25.653101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:53401 (local address 127.30.54.126:34041)
0808 07:11:25.653235 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:25.653237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:25.653316 (+ 79us) server_negotiation.cc:407] Connection header received
0808 07:11:25.653448 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:25.653449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:25.653463 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:25.653536 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:25.653973 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.654515 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.655282 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.655480 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.655535 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:25.658103 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:53401: BlockingRecv error: recv got EOF from 127.30.54.65:53401 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
Aug 08 07:11:25 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:25 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:25.790652 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:25.781069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:41927)
0808 07:11:25.781210 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:25.781221 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:25.781278 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:25.781415 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:25.781415 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:25.781582 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:25.781638 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:25.781640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.782394 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.782395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:25.783024 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:25.783027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.783396 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.783397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:25.783456 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:25.788690 (+ 5234us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:25.788696 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:25.790553 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
W20260808 07:11:25.790699 11794 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:25.781143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:41927 (local address 127.30.54.126:34041)
0808 07:11:25.781214 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:25.781216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:25.781277 (+ 61us) server_negotiation.cc:407] Connection header received
0808 07:11:25.781336 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:25.781337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:25.781347 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:25.781390 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:25.781706 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.782309 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.783112 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.783321 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.783397 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:25.790635 (+ 7238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:41927: BlockingRecv error: recv got EOF from 127.30.54.66:41927 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 07:11:25 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:25 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:25.908390 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:25.903700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:43893)
0808 07:11:25.903879 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:25.903891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:25.903990 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:25.904120 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:25.904121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:25.904308 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:25.904356 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:25.904358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.905044 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.905046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:25.905577 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:25.905579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.905946 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.905947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:25.906020 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:25.906441 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:25.906445 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:25.908295 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
W20260808 07:11:25.908432 11794 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:25.903788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:43893 (local address 127.30.54.126:34041)
0808 07:11:25.903860 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:25.903862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:25.903963 (+ 101us) server_negotiation.cc:407] Connection header received
0808 07:11:25.904047 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:25.904048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:25.904061 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:25.904116 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:25.904436 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.904951 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.905662 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:25.905857 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:25.905943 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:25.908370 (+ 2427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:43893: BlockingRecv error: recv got EOF from 127.30.54.67:43893 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 07:11:26 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:26 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:26.663401 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:26.658589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:50467)
0808 07:11:26.658732 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:26.658746 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:26.658818 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:26.659022 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:26.659023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:26.659197 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:26.659249 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:26.659251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.660151 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.660156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:26.660728 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:26.660730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.661079 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.661079 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:26.661120 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:26.661553 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:26.661557 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:26.663336 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
W20260808 07:11:26.663656 11802 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:26.658688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:50467 (local address 127.30.54.126:34041)
0808 07:11:26.658880 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:26.658882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:26.658891 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:26.658915 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:26.658916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:26.658931 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:26.659000 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:26.659317 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.660068 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.660804 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.661011 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.661066 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:26.663596 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:50467: BlockingRecv error: recv got EOF from 127.30.54.65:50467 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
Aug 08 07:11:26 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:26 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:26.795747 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:26.791179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:36677)
0808 07:11:26.791335 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:26.791347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:26.791414 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:26.791552 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:26.791553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:26.791723 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:26.791774 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:26.791775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.792431 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.792432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:26.792962 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:26.792964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.793280 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.793281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:26.793321 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:26.793827 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:26.793837 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:26.795690 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
W20260808 07:11:26.795887 11802 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:26.791248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:36677 (local address 127.30.54.126:34041)
0808 07:11:26.791319 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:26.791321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:26.791415 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:11:26.791459 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:26.791459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:26.791471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:26.791525 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:26.791836 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.792364 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.793032 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.793215 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.793264 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:26.795803 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:36677: BlockingRecv error: recv got EOF from 127.30.54.66:36677 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 07:11:26 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:26 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:26.913664 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:26.908886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:50545)
0808 07:11:26.909042 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:26.909053 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:26.909109 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:26.909252 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:26.909252 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:26.909451 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:26.909503 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:26.909505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.910140 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.910141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:26.910695 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:26.910697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.911194 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.911195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:26.911248 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:26.911819 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:26.911824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:26.913604 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":59,"threads_started":1}
W20260808 07:11:26.913795 11802 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:26.909056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:50545 (local address 127.30.54.126:34041)
0808 07:11:26.909117 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:26.909119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:26.909128 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:26.909158 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:26.909159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:26.909169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:26.909217 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:26.909570 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.910069 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.910910 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:26.911104 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:26.911166 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:26.913687 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:50545: BlockingRecv error: recv got EOF from 127.30.54.67:50545 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 07:11:27 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:27 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:27.668447 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:27.663890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:40949)
0808 07:11:27.664050 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:27.664060 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:27.664108 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:27.664241 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:27.664241 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:27.664413 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:27.664468 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.664470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.665253 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.665253 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.665798 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.665799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.666135 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.666136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.666176 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.666611 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:27.666615 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:27.668381 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
W20260808 07:11:27.668576 11806 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:27.663968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:40949 (local address 127.30.54.126:34041)
0808 07:11:27.664103 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:27.664105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:27.664112 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:11:27.664152 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:27.664153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:27.664163 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:27.664236 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:27.664564 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.665180 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.665871 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.666068 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.666124 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.668474 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:40949: BlockingRecv error: recv got EOF from 127.30.54.65:40949 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
Aug 08 07:11:27 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:27 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:27.800925 11807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:27.796246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:34195)
0808 07:11:27.796446 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:27.796457 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:27.796515 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:27.796630 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:27.796630 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:27.796830 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:27.796880 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.796882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.797516 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.797517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.798083 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.798085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.798390 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.798390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.798428 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.798865 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:27.798869 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:27.800858 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":136,"threads_started":1}
W20260808 07:11:27.801024 11806 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:27.796331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:34195 (local address 127.30.54.126:34041)
0808 07:11:27.796384 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:27.796386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:27.796528 (+ 142us) server_negotiation.cc:407] Connection header received
0808 07:11:27.796558 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:27.796558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:27.796569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:27.796618 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:27.796941 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.797431 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.798156 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.798330 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.798377 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.800959 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:34195: BlockingRecv error: recv got EOF from 127.30.54.66:34195 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 07:11:27 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:27 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:27.919240 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:27.914118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:55669)
0808 07:11:27.914419 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:27.914433 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:27.914527 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:27.914675 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:27.914676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:27.914950 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:27.915000 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.915002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.915688 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.915689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.916244 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:27.916246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.916641 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.916642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:27.916706 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.917260 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:27.917265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:27.919174 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":120,"threads_started":1}
W20260808 07:11:27.919370 11806 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:27.914218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:55669 (local address 127.30.54.126:34041)
0808 07:11:27.914424 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:27.914426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:27.914506 (+ 80us) server_negotiation.cc:407] Connection header received
0808 07:11:27.914582 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:27.914583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:27.914599 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:27.914667 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:27.915082 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.915593 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.916331 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:27.916533 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:27.916641 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:27.919268 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:55669: BlockingRecv error: recv got EOF from 127.30.54.67:55669 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
Aug 08 07:11:28 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:28 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:28.673842 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:28.668905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:53549)
0808 07:11:28.669091 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:28.669104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:28.669181 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:28.669365 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:28.669367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:28.669604 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:28.669666 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.669668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.670320 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.670320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.670918 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.670919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.671293 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.671293 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.671334 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.671835 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:28.671840 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:28.673784 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
W20260808 07:11:28.674011 11810 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:28.669020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:53549 (local address 127.30.54.126:34041)
0808 07:11:28.669178 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:28.669181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:28.669190 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:28.669257 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:28.669258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:28.669271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:28.669345 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:28.669743 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.670253 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.670995 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.671221 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.671279 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.673913 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:53549: BlockingRecv error: recv got EOF from 127.30.54.65:53549 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
Aug 08 07:11:28 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:28 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:28.806098 11811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:28.801561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:47005)
0808 07:11:28.801718 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:28.801747 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:28.801822 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:28.801949 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:28.801949 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:28.802113 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:28.802162 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.802163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.802774 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.802775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.803347 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.803350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.803692 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.803693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.803737 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.804177 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:28.804183 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:28.806041 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
W20260808 07:11:28.806200 11810 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:28.801624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:47005 (local address 127.30.54.126:34041)
0808 07:11:28.801728 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:28.801730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:28.801818 (+ 88us) server_negotiation.cc:407] Connection header received
0808 07:11:28.801887 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:28.801888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:28.801900 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:28.801944 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:28.802222 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.802705 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.803424 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.803620 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.803676 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.806127 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:47005: BlockingRecv error: recv got EOF from 127.30.54.66:47005 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 07:11:28 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:28 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:28.924103 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:28.919700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:53427)
0808 07:11:28.919880 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:28.919890 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:28.919954 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:28.920079 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:28.920080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:28.920245 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:28.920295 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.920297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.920923 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.920923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.921448 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:28.921449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.921767 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.921768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:28.921816 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.922317 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:28.922322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:28.924045 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
W20260808 07:11:28.924199 11810 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:28.919813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:53427 (local address 127.30.54.126:34041)
0808 07:11:28.919868 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:28.919870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:28.919952 (+ 82us) server_negotiation.cc:407] Connection header received
0808 07:11:28.919998 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:28.919998 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:28.920011 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:28.920058 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:28.920354 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.920858 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.921525 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:28.921702 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:28.921752 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:28.924133 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:53427: BlockingRecv error: recv got EOF from 127.30.54.67:53427 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 07:11:29 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:29 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:29.679229 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:29.674304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:47525)
0808 07:11:29.674475 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:29.674486 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:29.674587 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:29.674747 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:29.674748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:29.674939 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:29.674989 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:29.674991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.675837 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.675839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:29.676484 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:29.676487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.676833 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.676834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:29.676878 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.677384 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:29.677389 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:29.679162 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
W20260808 07:11:29.679329 11815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:29.674397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:47525 (local address 127.30.54.126:34041)
0808 07:11:29.674549 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:29.674551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:29.674565 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:11:29.674657 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:29.674658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:29.674668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:29.674728 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:29.675057 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.675758 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.676566 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.676767 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.676820 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.679255 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:47525: BlockingRecv error: recv got EOF from 127.30.54.65:47525 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
Aug 08 07:11:29 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:29 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:29.811414 11820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:29.806602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:47543)
0808 07:11:29.806777 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:29.806787 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:29.806901 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:29.807043 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:29.807044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:29.807209 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:29.807258 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:29.807260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.807893 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.807894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:29.808626 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:29.808632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.809053 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.809054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:29.809115 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.809540 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:29.809545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:29.811322 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
W20260808 07:11:29.811489 11815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:29.806741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:47543 (local address 127.30.54.126:34041)
0808 07:11:29.806833 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:29.806835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:29.806903 (+ 68us) server_negotiation.cc:407] Connection header received
0808 07:11:29.806949 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:29.806949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:29.806961 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:29.807012 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:29.807322 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.807829 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.808700 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.808979 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.809055 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.811414 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:47543: BlockingRecv error: recv got EOF from 127.30.54.66:47543 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 07:11:29 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:29 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:29.929602 11821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:29.924596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:36807)
0808 07:11:29.924784 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:29.924801 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:29.924885 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:29.925073 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:29.925074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:29.925257 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:29.925312 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:29.925314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.926053 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.926054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:29.926635 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:29.926637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.927116 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.927117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:29.927184 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.927650 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:29.927655 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:29.929545 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":117,"threads_started":1}
W20260808 07:11:29.929723 11815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:29.924676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:36807 (local address 127.30.54.126:34041)
0808 07:11:29.924761 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:29.924763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:29.924863 (+ 100us) server_negotiation.cc:407] Connection header received
0808 07:11:29.924959 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:29.924960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:29.924975 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:29.925061 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:29.925393 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.925973 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.926721 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:29.927026 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:29.927114 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:29.929642 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:36807: BlockingRecv error: recv got EOF from 127.30.54.67:36807 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 07:11:30 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:30 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:30.684476 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:30.679729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:42123)
0808 07:11:30.679881 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:30.679891 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:30.679940 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:30.680077 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:30.680077 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:30.680256 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:30.680308 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.680310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.680968 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.680969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.681509 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.681511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.681863 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.681869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.681919 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.682469 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:30.682473 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:30.684380 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
W20260808 07:11:30.684540 11825 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:30.679821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:42123 (local address 127.30.54.126:34041)
0808 07:11:30.679948 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:30.679949 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:30.679959 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:30.679983 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:30.679983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:30.679995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:30.680058 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:30.680374 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.680890 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.681589 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.681788 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.681862 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.684469 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:42123: BlockingRecv error: recv got EOF from 127.30.54.65:42123 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
Aug 08 07:11:30 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:30 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:30.821485 11828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:30.811886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:52161)
0808 07:11:30.812063 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:30.812075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:30.812161 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:30.812320 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:30.812321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:30.812531 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:30.812580 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.812582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.813277 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.813278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.813830 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.813832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.814199 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.814200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.814261 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.819581 (+ 5320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:30.819586 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:30.821363 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":97,"threads_started":1}
W20260808 07:11:30.821579 11825 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:30.811964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:52161 (local address 127.30.54.126:34041)
0808 07:11:30.812050 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:30.812052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:30.812142 (+ 90us) server_negotiation.cc:407] Connection header received
0808 07:11:30.812220 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:30.812221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:30.812234 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:30.812313 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:30.812671 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.813190 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.813923 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.814122 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.814201 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.821510 (+ 7309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:52161: BlockingRecv error: recv got EOF from 127.30.54.66:52161 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 07:11:30 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:30 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.67@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:30.934597 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:30.930075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.67:34823)
0808 07:11:30.930209 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:30.930220 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:30.930280 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:30.930403 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:30.930404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:30.930579 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:30.930629 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.930630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.931338 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.931339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.931884 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:30.931886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.932194 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.932194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:30.932234 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.932684 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:30.932688 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:30.934525 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":56,"threads_started":1}
W20260808 07:11:30.934660 11825 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:30.930135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:34823 (local address 127.30.54.126:34041)
0808 07:11:30.930212 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:30.930214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:30.930283 (+ 69us) server_negotiation.cc:407] Connection header received
0808 07:11:30.930321 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:30.930322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:30.930335 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:30.930384 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:30.930693 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.931252 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.931957 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:30.932134 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:30.932183 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:30.934614 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.67:34823: BlockingRecv error: recv got EOF from 127.30.54.67:34823 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 07:11:31 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:31 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.65@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:31.689266 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.684986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.65:59327)
0808 07:11:31.685107 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.685120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.685184 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.685327 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.685327 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.685495 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:31.685548 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.685549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.686182 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.686183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.686712 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.686713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.687055 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.687056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.687102 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.687514 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:31.687519 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:31.689198 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
W20260808 07:11:31.689363 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:31.685058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:59327 (local address 127.30.54.126:34041)
0808 07:11:31.685205 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.685207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.685214 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:11:31.685237 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.685238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.685249 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.685307 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.685614 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.686117 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.686781 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.686994 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.687043 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.689288 (+ 2245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.65:59327: BlockingRecv error: recv got EOF from 127.30.54.65:59327 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
Aug 08 07:11:31 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:11:31 dist-test-slave-znh6 krb5kdc[10199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.54.66@KRBTEST.COM for oryx/127.30.54.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:11:31.827031 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:31.822114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34041 (local address 127.30.54.66:60449)
0808 07:11:31.822309 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:31.822320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:31.822376 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:31.822507 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:31.822508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:31.822675 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:31.822727 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.822729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.823475 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.823476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.824000 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:31.824002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.824458 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.824459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:31.824545 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.824969 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:31.824974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:31.826933 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.54.126:34041: Server oryx/127.30.54.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
W20260808 07:11:31.827069 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:31.822233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:60449 (local address 127.30.54.126:34041)
0808 07:11:31.822298 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:31.822300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:31.822380 (+ 80us) server_negotiation.cc:407] Connection header received
0808 07:11:31.822424 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:31.822425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:31.822436 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:31.822494 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:31.822791 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.823394 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.824094 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:31.824376 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:31.824456 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:31.827010 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.54.66:60449: BlockingRecv error: recv got EOF from 127.30.54.66:60449 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:11:31.856220 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 11087
I20260808 07:11:31.867244 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 11332
I20260808 07:11:31.873919 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 11531
I20260808 07:11:31.880627 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 10981
2026-08-08T07:11:31Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17664 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T07:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:31Z Disabled control of system clock
I20260808 07:11:31.913681 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:38629
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:37411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:38629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:11:31.994607 11842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:31.994836 11842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:31.994859 11842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:31.995959 11842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:31.995997 11842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:31.996011 11842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:31.996023 11842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:31.996035 11842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:31.997334 11842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:38629
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:38629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:31.997553 11842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:31.997786 11842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:32.003366 11842 server_base.cc:1034] running on GCE node
W20260808 07:11:32.003474 11855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.003352 11849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.003329 11850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.003865 11842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.004189 11842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.005364 11842 hybrid_clock.cc:648] HybridClock initialized: now 1786173092005352 us; error 31 us; skew 500 ppm
I20260808 07:11:32.006625 11842 webserver.cc:460] Webserver started at http://127.30.54.126:39749/ using document root <none> and password file <none>
I20260808 07:11:32.006927 11842 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.007001 11842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.007164 11842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.008229 11842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "778e2d6ee2174a0bbd1c285e37592efd"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "0c1ecc87b7672f7c6c141e44ec0c7dc0"
server_key_iv: "e3e66c97c1ae774a258e28a84462b290"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.008589 11842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "778e2d6ee2174a0bbd1c285e37592efd"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "0c1ecc87b7672f7c6c141e44ec0c7dc0"
server_key_iv: "e3e66c97c1ae774a258e28a84462b290"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.009897 11842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:11:32.010568 11864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.010738 11842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:32.010859 11842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "778e2d6ee2174a0bbd1c285e37592efd"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "0c1ecc87b7672f7c6c141e44ec0c7dc0"
server_key_iv: "e3e66c97c1ae774a258e28a84462b290"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.010946 11842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.066039 11842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.066406 11842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.066578 11842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.070566 11842 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:38629
I20260808 07:11:32.070641 11942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:38629 every 8 connection(s)
I20260808 07:11:32.071079 11842 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:32.071768 11943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:32.074357 11943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd: Bootstrap starting.
I20260808 07:11:32.074977 11943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:32.075227 11943 log.cc:824] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:32.075873 11943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd: No bootstrap required, opened a new log
I20260808 07:11:32.077396 11943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778e2d6ee2174a0bbd1c285e37592efd" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38629 } }
I20260808 07:11:32.077528 11943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:32.077551 11943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 778e2d6ee2174a0bbd1c285e37592efd, State: Initialized, Role: FOLLOWER
I20260808 07:11:32.077683 11943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778e2d6ee2174a0bbd1c285e37592efd" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38629 } }
I20260808 07:11:32.077762 11943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:32.077785 11943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:32.077816 11943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:32.078282 11943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778e2d6ee2174a0bbd1c285e37592efd" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38629 } }
I20260808 07:11:32.078390 11943 leader_election.cc:302] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 778e2d6ee2174a0bbd1c285e37592efd; no voters:
I20260808 07:11:32.078567 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 11842
I20260808 07:11:32.078572 11943 leader_election.cc:288] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:32.078599 11947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:32.078676 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:11:32.078727 11947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [term 1 LEADER]: Becoming Leader. State: Replica: 778e2d6ee2174a0bbd1c285e37592efd, State: Running, Role: LEADER
I20260808 07:11:32.078840 11947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778e2d6ee2174a0bbd1c285e37592efd" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38629 } }
I20260808 07:11:32.078877 11943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:32.078903 30937 external_mini_cluster.cc:1428] Setting key 2634e6ad9d4d0556463e346ec62657ea
I20260808 07:11:32.079135 11947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 778e2d6ee2174a0bbd1c285e37592efd. Latest consensus state: current_term: 1 leader_uuid: "778e2d6ee2174a0bbd1c285e37592efd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778e2d6ee2174a0bbd1c285e37592efd" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38629 } } }
I20260808 07:11:32.079309 11947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.079695 11948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "778e2d6ee2174a0bbd1c285e37592efd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778e2d6ee2174a0bbd1c285e37592efd" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38629 } } }
I20260808 07:11:32.079835 11948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.079746 11962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:32.080403 11962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:32.081998 11962 catalog_manager.cc:1347] Generated new cluster ID: a0bce5270bf841b5adce10983ab907e3
I20260808 07:11:32.082055 11962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:32.085330 11965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.079522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53418 (local address 127.30.54.126:38629)
0808 07:11:32.080455 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.080463 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.080724 (+ 261us) server_negotiation.cc:407] Connection header received
0808 07:11:32.081010 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.081017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.081108 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.081250 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.081525 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.082532 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.083448 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.083879 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.084013 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.084709 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.084713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.084713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.084885 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.084918 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.084928 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.085003 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.085092 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.085169 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":97,"threads_started":1}
I20260808 07:11:32.113626 11962 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:32.114362 11962 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:32.117692 11962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 778e2d6ee2174a0bbd1c285e37592efd: Generated new TSK 0
I20260808 07:11:32.117865 11962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:11:32.124060 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38629
--builtin_ntp_servers=127.30.54.84:37411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 07:11:32.207466 11976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.207657 11976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.207679 11976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.208807 11976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.208849 11976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.208873 11976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:11:32.210184 11976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:32.210419 11976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.210716 11976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:32.215943 11982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.215917 11986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.215937 11981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.216207 11976 server_base.cc:1034] running on GCE node
I20260808 07:11:32.216425 11976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.216663 11976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.217809 11976 hybrid_clock.cc:648] HybridClock initialized: now 1786173092217792 us; error 41 us; skew 500 ppm
I20260808 07:11:32.219098 11976 webserver.cc:460] Webserver started at http://127.30.54.65:44191/ using document root <none> and password file <none>
I20260808 07:11:32.219375 11976 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.219424 11976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.219589 11976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.220582 11976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "455f44ef84c04c7ba96a5e7390c064b6"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "aa95638e5607d63329b76138290d8900"
server_key_iv: "b56be07e052b91b4e82d09ea47fbfdf9"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.220937 11976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "455f44ef84c04c7ba96a5e7390c064b6"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "aa95638e5607d63329b76138290d8900"
server_key_iv: "b56be07e052b91b4e82d09ea47fbfdf9"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.222270 11976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:32.223163 11992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.223369 11976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:32.223460 11976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "455f44ef84c04c7ba96a5e7390c064b6"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "aa95638e5607d63329b76138290d8900"
server_key_iv: "b56be07e052b91b4e82d09ea47fbfdf9"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.223534 11976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.239555 11976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.239838 11976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.239974 11976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.240203 11976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:32.240535 11976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:32.240571 11976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.240590 11976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:32.240624 11976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.246670 11976 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:33121
I20260808 07:11:32.246745 12166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:33121 every 8 connection(s)
I20260808 07:11:32.247049 11976 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:32.249831 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 11976
I20260808 07:11:32.249923 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:32.250123 30937 external_mini_cluster.cc:1428] Setting key 80bf49a47c2dfc19039d4b120327a32a
I20260808 07:11:32.251103 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38629
--builtin_ntp_servers=127.30.54.84:37411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 07:11:32.253008 11965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.247818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:48571 (local address 127.30.54.126:38629)
0808 07:11:32.247890 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.247892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.248132 (+ 240us) server_negotiation.cc:407] Connection header received
0808 07:11:32.248676 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.248677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.248692 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.248749 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.249411 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.250171 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.251398 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.251697 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.251776 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.252638 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.252640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.252640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.252645 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.252660 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.252666 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.252725 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.252868 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.252903 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:11:32.253212 12171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.247813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:38629 (local address 127.30.54.65:48571)
0808 07:11:32.248003 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.248041 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.248561 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.248812 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.248816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.248831 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:32.249310 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.249314 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.250282 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.250284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.251287 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.251290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.251808 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.251809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.251928 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.252486 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:32.252487 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:32.252514 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:32.252536 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:32.252758 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:32.252764 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:32.252767 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:32.252847 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:11:32.252971 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260808 07:11:32.253700 12167 heartbeater.cc:347] Connected to a master server at 127.30.54.126:38629
I20260808 07:11:32.253808 12167 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:32.253980 12167 heartbeater.cc:511] Master 127.30.54.126:38629 requested a full tablet report, sending...
I20260808 07:11:32.254506 11891 ts_manager.cc:194] Registered new tserver with Master: 455f44ef84c04c7ba96a5e7390c064b6 (127.30.54.65:33121)
I20260808 07:11:32.255126 11891 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.65:48571
W20260808 07:11:32.334689 12173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.334966 12173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.335004 12173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.336110 12173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.336171 12173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.336215 12173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:11:32.337576 12173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:32.337823 12173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.338111 12173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:32.343308 12173 server_base.cc:1034] running on GCE node
W20260808 07:11:32.343254 12180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.343289 12183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.343333 12178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.343737 12173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.344014 12173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.345170 12173 hybrid_clock.cc:648] HybridClock initialized: now 1786173092345154 us; error 34 us; skew 500 ppm
I20260808 07:11:32.346563 12173 webserver.cc:460] Webserver started at http://127.30.54.66:43109/ using document root <none> and password file <none>
I20260808 07:11:32.346871 12173 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.346961 12173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.347101 12173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.348133 12173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "64c2150ce5c84e10869181902fac17ba"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "436cd380cad84339a3297b63ce943466"
server_key_iv: "56c6a9869c08c3b70955165fdb288424"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.348500 12173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "64c2150ce5c84e10869181902fac17ba"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "436cd380cad84339a3297b63ce943466"
server_key_iv: "56c6a9869c08c3b70955165fdb288424"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.349777 12173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:32.350478 12191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.350652 12173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:32.350752 12173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "64c2150ce5c84e10869181902fac17ba"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "436cd380cad84339a3297b63ce943466"
server_key_iv: "56c6a9869c08c3b70955165fdb288424"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.350867 12173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.376158 12173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.376487 12173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.376647 12173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.376896 12173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:32.377243 12173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:32.377302 12173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.377337 12173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:32.377381 12173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.383663 12173 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:37759
I20260808 07:11:32.383728 12372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:37759 every 8 connection(s)
I20260808 07:11:32.384079 12173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:32.387583 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 12173
I20260808 07:11:32.387688 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:32.387892 30937 external_mini_cluster.cc:1428] Setting key 6946f9aae0f2691389035149e4be1e4c
I20260808 07:11:32.388936 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38629
--builtin_ntp_servers=127.30.54.84:37411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 07:11:32.390064 11965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.384873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:40207 (local address 127.30.54.126:38629)
0808 07:11:32.384942 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.384944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.385190 (+ 246us) server_negotiation.cc:407] Connection header received
0808 07:11:32.385729 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.385731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.385742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.385798 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.386488 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.387028 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.388359 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.388630 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.388721 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.389765 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.389766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.389766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.389771 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.389787 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.389792 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.389847 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.389952 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.389984 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:11:32.390283 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.384853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:38629 (local address 127.30.54.66:40207)
0808 07:11:32.385064 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.385097 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.385616 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.385868 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.385876 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.385885 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:32.386385 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.386389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.387132 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.387133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.388271 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.388274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.388730 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.388731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.388885 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.389574 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:32.389575 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:32.389603 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:32.389628 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:32.389855 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:32.389860 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:32.389863 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:32.389955 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:11:32.390054 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260808 07:11:32.390869 12373 heartbeater.cc:347] Connected to a master server at 127.30.54.126:38629
I20260808 07:11:32.390978 12373 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:32.391177 12373 heartbeater.cc:511] Master 127.30.54.126:38629 requested a full tablet report, sending...
I20260808 07:11:32.391551 11891 ts_manager.cc:194] Registered new tserver with Master: 64c2150ce5c84e10869181902fac17ba (127.30.54.66:37759)
I20260808 07:11:32.391964 11891 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.66:40207
W20260808 07:11:32.474876 12377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.475060 12377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.475080 12377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.476186 12377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.476226 12377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.476250 12377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:11:32.477535 12377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:32.477751 12377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.478044 12377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:32.483443 12385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.483503 12377 server_base.cc:1034] running on GCE node
W20260808 07:11:32.483484 12388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.483484 12384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.483851 12377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.484092 12377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.485230 12377 hybrid_clock.cc:648] HybridClock initialized: now 1786173092485224 us; error 31 us; skew 500 ppm
I20260808 07:11:32.486512 12377 webserver.cc:460] Webserver started at http://127.30.54.67:36007/ using document root <none> and password file <none>
I20260808 07:11:32.486752 12377 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.486799 12377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.487007 12377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.488032 12377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "7714b33e3ee842f7841808f948f2f952"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "c4921eadc9b586daf9c47aec77fe69d9"
server_key_iv: "3692268ebc708b0a9b8fde6fe47f488f"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.488374 12377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "7714b33e3ee842f7841808f948f2f952"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "c4921eadc9b586daf9c47aec77fe69d9"
server_key_iv: "3692268ebc708b0a9b8fde6fe47f488f"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.489729 12377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:32.490509 12394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.490777 12377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:32.490935 12377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "7714b33e3ee842f7841808f948f2f952"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "c4921eadc9b586daf9c47aec77fe69d9"
server_key_iv: "3692268ebc708b0a9b8fde6fe47f488f"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.491014 12377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.499876 12377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.500129 12377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.500269 12377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.500476 12377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:32.500793 12377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:32.500856 12377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.500895 12377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:32.500921 12377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.506953 12377 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:39453
I20260808 07:11:32.507005 12567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:39453 every 8 connection(s)
I20260808 07:11:32.507295 12377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:32.512176 11965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.507989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:44845 (local address 127.30.54.126:38629)
0808 07:11:32.508058 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.508060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.508279 (+ 219us) server_negotiation.cc:407] Connection header received
0808 07:11:32.508727 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.508729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.508740 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.508794 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.509383 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.509889 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.510781 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.511118 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.511174 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.511960 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.511961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.511961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.511965 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.511976 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.511980 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.512018 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.512086 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.512109 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:11:32.512349 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.507977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:38629 (local address 127.30.54.67:44845)
0808 07:11:32.508159 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.508197 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.508623 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.508853 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.508856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.508863 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:32.509310 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.509314 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.509969 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.509970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.510711 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:32.510713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:32.511202 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:32.511203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:32.511304 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:32.511821 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:32.511822 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:32.511859 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:32.511879 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:32.512020 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:32.512022 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:32.512024 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:32.512069 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:11:32.512138 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260808 07:11:32.512908 12568 heartbeater.cc:347] Connected to a master server at 127.30.54.126:38629
I20260808 07:11:32.512997 12568 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:32.513125 12568 heartbeater.cc:511] Master 127.30.54.126:38629 requested a full tablet report, sending...
I20260808 07:11:32.513473 11891 ts_manager.cc:194] Registered new tserver with Master: 7714b33e3ee842f7841808f948f2f952 (127.30.54.67:39453)
I20260808 07:11:32.513829 11891 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.67:44845
I20260808 07:11:32.514961 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 12377
I20260808 07:11:32.515066 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:32.515254 30937 external_mini_cluster.cc:1428] Setting key eeb83487e39facf0d3ee50c65dd443f3
I20260808 07:11:32.515996 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:11:32.517827 11965 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:32.517326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.30.54.126:38629)
0808 07:11:32.517382 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.517384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.517511 (+ 127us) server_negotiation.cc:407] Connection header received
0808 07:11:32.517586 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.517587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.517597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.517642 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.517771 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53432: BlockingRecv error: recv got EOF from 127.0.0.1:53432 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:11:32.517937 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 11976
I20260808 07:11:32.537319 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 12173
I20260808 07:11:32.544627 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 12377
I20260808 07:11:32.555186 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 11842
2026-08-08T07:11:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (673 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T07:11:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:32Z Disabled control of system clock
I20260808 07:11:32.587159 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:35539
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:35539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:11:32.668542 12584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.668735 12584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.668761 12584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.669866 12584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:32.669907 12584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.669920 12584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.669966 12584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:32.669993 12584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:32.671370 12584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:35539
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:35539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:32.671597 12584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.671885 12584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:32.676664 12595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.676657 12596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.676657 12598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.676893 12584 server_base.cc:1034] running on GCE node
I20260808 07:11:32.677137 12584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.677404 12584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.678582 12584 hybrid_clock.cc:648] HybridClock initialized: now 1786173092678568 us; error 30 us; skew 500 ppm
I20260808 07:11:32.679872 12584 webserver.cc:460] Webserver started at http://127.30.54.126:39843/ using document root <none> and password file <none>
I20260808 07:11:32.680126 12584 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.680197 12584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.680357 12584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.681514 12584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "779d5026f74042a2902f97d93da7dbc6"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "c7c010ff56c7fb77fb667c0df103bfc9"
server_key_iv: "f731e2c2e373ae1d844db2c0153f1fc8"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.681878 12584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "779d5026f74042a2902f97d93da7dbc6"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "c7c010ff56c7fb77fb667c0df103bfc9"
server_key_iv: "f731e2c2e373ae1d844db2c0153f1fc8"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.683193 12584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:32.683888 12605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.684038 12584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:32.684135 12584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "779d5026f74042a2902f97d93da7dbc6"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "c7c010ff56c7fb77fb667c0df103bfc9"
server_key_iv: "f731e2c2e373ae1d844db2c0153f1fc8"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.684213 12584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.687477 12584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.687712 12584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.687862 12584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.691686 12584 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:35539
I20260808 07:11:32.691716 12692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:35539 every 8 connection(s)
I20260808 07:11:32.692028 12584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:32.692631 12694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:32.695148 12694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6: Bootstrap starting.
I20260808 07:11:32.695640 12694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:32.695868 12694 log.cc:824] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:32.696494 12694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6: No bootstrap required, opened a new log
I20260808 07:11:32.698467 12694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d5026f74042a2902f97d93da7dbc6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35539 } }
I20260808 07:11:32.698591 12694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:32.698613 12694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779d5026f74042a2902f97d93da7dbc6, State: Initialized, Role: FOLLOWER
I20260808 07:11:32.698751 12694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d5026f74042a2902f97d93da7dbc6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35539 } }
I20260808 07:11:32.698880 12694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:32.698920 12694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:32.698954 12694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:32.699419 12694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d5026f74042a2902f97d93da7dbc6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35539 } }
I20260808 07:11:32.699501 12694 leader_election.cc:302] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 779d5026f74042a2902f97d93da7dbc6; no voters:
I20260808 07:11:32.699612 12694 leader_election.cc:288] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:32.699699 12698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:32.699887 12698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 779d5026f74042a2902f97d93da7dbc6, State: Running, Role: LEADER
I20260808 07:11:32.699891 12694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:32.699985 12698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d5026f74042a2902f97d93da7dbc6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35539 } }
I20260808 07:11:32.700381 12701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 779d5026f74042a2902f97d93da7dbc6. Latest consensus state: current_term: 1 leader_uuid: "779d5026f74042a2902f97d93da7dbc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d5026f74042a2902f97d93da7dbc6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35539 } } }
I20260808 07:11:32.700543 12701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.700562 12699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "779d5026f74042a2902f97d93da7dbc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d5026f74042a2902f97d93da7dbc6" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 35539 } } }
I20260808 07:11:32.700704 12699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:32.701018 12709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:32.701354 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 12584
I20260808 07:11:32.701449 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:11:32.701527 12709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:32.701686 30937 external_mini_cluster.cc:1428] Setting key edea3ad57cedd15dd14c5627db2995e3
I20260808 07:11:32.703629 12709 catalog_manager.cc:1347] Generated new cluster ID: 555b322ebebc459cb6a88aa75ec661b5
I20260808 07:11:32.703691 12709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:32.704049 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.702316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.30.54.126:35539)
0808 07:11:32.702518 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.702529 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.702552 (+ 23us) server_negotiation.cc:407] Connection header received
0808 07:11:32.702664 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.702671 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.702718 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.702888 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.703516 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.703519 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.703520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.703525 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.703574 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.703590 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.703798 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.703828 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.703894 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260808 07:11:32.718468 12709 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:32.718626 12709 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:32.723681 12709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 779d5026f74042a2902f97d93da7dbc6: Generated new TSK 0
I20260808 07:11:32.723877 12709 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:11:32.731032 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35539
--builtin_ntp_servers=127.30.54.84:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:11:32.816679 12728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.816859 12728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.816880 12728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.818073 12728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.818114 12728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.818140 12728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:11:32.819525 12728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:32.819747 12728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.820043 12728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:32.826078 12735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.826217 12737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.826084 12734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.826570 12728 server_base.cc:1034] running on GCE node
I20260808 07:11:32.826766 12728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.827059 12728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.828220 12728 hybrid_clock.cc:648] HybridClock initialized: now 1786173092828208 us; error 28 us; skew 500 ppm
I20260808 07:11:32.829491 12728 webserver.cc:460] Webserver started at http://127.30.54.65:46143/ using document root <none> and password file <none>
I20260808 07:11:32.829754 12728 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.829814 12728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.829973 12728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.831204 12728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "933fe0ac4a014a4997c80a175029fb74"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "5e726887fb69aa71bf241ef078d1793a"
server_key_iv: "f7882f9d32beef6ad5eef07529bce22f"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.831566 12728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "933fe0ac4a014a4997c80a175029fb74"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "5e726887fb69aa71bf241ef078d1793a"
server_key_iv: "f7882f9d32beef6ad5eef07529bce22f"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.833039 12728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:32.833833 12745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.834002 12728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:32.834091 12728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "933fe0ac4a014a4997c80a175029fb74"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "5e726887fb69aa71bf241ef078d1793a"
server_key_iv: "f7882f9d32beef6ad5eef07529bce22f"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.834146 12728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.837440 12728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.837648 12728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.837743 12728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.837908 12728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:32.838147 12728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:32.838178 12728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.838198 12728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:32.838210 12728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.844075 12728 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:41727
I20260808 07:11:32.844156 12911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:41727 every 8 connection(s)
I20260808 07:11:32.844480 12728 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:32.847090 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 12728
I20260808 07:11:32.847198 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:32.847481 30937 external_mini_cluster.cc:1428] Setting key 745842add143805b950e34da52fb5310
I20260808 07:11:32.847630 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.845226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:46899 (local address 127.30.54.126:35539)
0808 07:11:32.845296 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.845298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.845495 (+ 197us) server_negotiation.cc:407] Connection header received
0808 07:11:32.845966 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.845967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.845975 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.846042 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.847353 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.847355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.847355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.847361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.847386 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.847392 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.847472 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.847542 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.847559 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:11:32.847923 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.845153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:35539 (local address 127.30.54.65:46899)
0808 07:11:32.845362 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.845402 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.845849 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.846116 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.846119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.846127 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:32.847153 (+ 1026us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:32.847154 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:32.847182 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:32.847237 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:32.847469 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:32.847472 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:32.847474 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:32.847548 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:11:32.847613 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260808 07:11:32.848452 12913 heartbeater.cc:347] Connected to a master server at 127.30.54.126:35539
I20260808 07:11:32.848552 12913 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:32.848546 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35539
--builtin_ntp_servers=127.30.54.84:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 07:11:32.848713 12913 heartbeater.cc:511] Master 127.30.54.126:35539 requested a full tablet report, sending...
I20260808 07:11:32.849195 12636 ts_manager.cc:194] Registered new tserver with Master: 933fe0ac4a014a4997c80a175029fb74 (127.30.54.65:41727)
W20260808 07:11:32.929950 12918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:32.930140 12918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:32.930162 12918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:32.931329 12918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:32.931370 12918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:32.931402 12918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:11:32.932722 12918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:32.932937 12918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:32.933225 12918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:32.938369 12924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.938459 12918 server_base.cc:1034] running on GCE node
W20260808 07:11:32.938431 12927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:32.938383 12925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:32.938786 12918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:32.939066 12918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:32.940230 12918 hybrid_clock.cc:648] HybridClock initialized: now 1786173092940222 us; error 39 us; skew 500 ppm
I20260808 07:11:32.941533 12918 webserver.cc:460] Webserver started at http://127.30.54.66:39681/ using document root <none> and password file <none>
I20260808 07:11:32.941762 12918 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:32.941833 12918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:32.942003 12918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:32.943169 12918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "1d18d40e9aad486c9e317b1601647e77"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "b130dc2d42056c085fd3224f67c2b474"
server_key_iv: "bd9f02ceadd89373d9f21f4a58548f32"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.943533 12918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "1d18d40e9aad486c9e317b1601647e77"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "b130dc2d42056c085fd3224f67c2b474"
server_key_iv: "bd9f02ceadd89373d9f21f4a58548f32"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.944861 12918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:32.945583 12933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.945766 12918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:32.945866 12918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "1d18d40e9aad486c9e317b1601647e77"
format_stamp: "Formatted at 2026-08-08 07:11:32 on dist-test-slave-znh6"
server_key: "b130dc2d42056c085fd3224f67c2b474"
server_key_iv: "bd9f02ceadd89373d9f21f4a58548f32"
server_key_version: "encryptionkey@0"
I20260808 07:11:32.945938 12918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:32.949245 12918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:32.949497 12918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:32.949640 12918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:32.949836 12918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:32.950132 12918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:32.950186 12918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.950222 12918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:32.950250 12918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:32.956015 12918 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:46829
I20260808 07:11:32.956050 13096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:46829 every 8 connection(s)
I20260808 07:11:32.956378 12918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:32.959347 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.957080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:41113 (local address 127.30.54.126:35539)
0808 07:11:32.957145 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:32.957147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:32.957371 (+ 224us) server_negotiation.cc:407] Connection header received
0808 07:11:32.957858 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:32.957859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:32.957867 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:32.957923 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:32.959058 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:32.959060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:32.959060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:32.959065 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:32.959082 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:32.959086 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:32.959166 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:32.959257 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 07:11:32.959272 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:11:32.959565 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:32.957045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:35539 (local address 127.30.54.66:41113)
0808 07:11:32.957251 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:32.957281 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:32.957734 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:32.958001 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:32.958004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:32.958011 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:32.958914 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:32.958914 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:32.958941 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:32.958966 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:32.959162 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:32.959167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:32.959171 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:11:32.959268 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 07:11:32.959325 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260808 07:11:32.960047 13097 heartbeater.cc:347] Connected to a master server at 127.30.54.126:35539
I20260808 07:11:32.960119 13097 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:32.960218 13097 heartbeater.cc:511] Master 127.30.54.126:35539 requested a full tablet report, sending...
I20260808 07:11:32.960518 12636 ts_manager.cc:194] Registered new tserver with Master: 1d18d40e9aad486c9e317b1601647e77 (127.30.54.66:46829)
I20260808 07:11:32.964059 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 12918
I20260808 07:11:32.964139 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:32.964349 30937 external_mini_cluster.cc:1428] Setting key 9b1af607682f462275f908654de89e5e
I20260808 07:11:32.965292 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35539
--builtin_ntp_servers=127.30.54.84:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:11:33.045608 13105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:33.045799 13105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:33.045820 13105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:33.046890 13105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:33.046923 13105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:33.046955 13105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:11:33.048190 13105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:35539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:33.048463 13105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:33.048766 13105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:33.053864 13118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:33.053840 13116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:33.053848 13113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:33.054148 13105 server_base.cc:1034] running on GCE node
I20260808 07:11:33.054364 13105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:33.054560 13105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:33.055694 13105 hybrid_clock.cc:648] HybridClock initialized: now 1786173093055684 us; error 29 us; skew 500 ppm
I20260808 07:11:33.056869 13105 webserver.cc:460] Webserver started at http://127.30.54.67:45835/ using document root <none> and password file <none>
I20260808 07:11:33.057117 13105 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:33.057165 13105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:33.057294 13105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:33.058312 13105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "d47fc3d833814aed80a5354a397b3f5c"
format_stamp: "Formatted at 2026-08-08 07:11:33 on dist-test-slave-znh6"
server_key: "f113dcdeed41d42f45d7bc8cbd3cb586"
server_key_iv: "014160a16d4a3b882153184c7c9ca81c"
server_key_version: "encryptionkey@0"
I20260808 07:11:33.058669 13105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "d47fc3d833814aed80a5354a397b3f5c"
format_stamp: "Formatted at 2026-08-08 07:11:33 on dist-test-slave-znh6"
server_key: "f113dcdeed41d42f45d7bc8cbd3cb586"
server_key_iv: "014160a16d4a3b882153184c7c9ca81c"
server_key_version: "encryptionkey@0"
I20260808 07:11:33.059993 13105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:11:33.060748 13126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:33.060930 13105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:33.061020 13105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "d47fc3d833814aed80a5354a397b3f5c"
format_stamp: "Formatted at 2026-08-08 07:11:33 on dist-test-slave-znh6"
server_key: "f113dcdeed41d42f45d7bc8cbd3cb586"
server_key_iv: "014160a16d4a3b882153184c7c9ca81c"
server_key_version: "encryptionkey@0"
I20260808 07:11:33.061093 13105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:33.064453 13105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:33.064689 13105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:33.064827 13105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:33.065032 13105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:33.065320 13105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:33.065351 13105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:33.065405 13105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:33.065441 13105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:33.071353 13105 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:35375
I20260808 07:11:33.071399 13300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:35375 every 8 connection(s)
I20260808 07:11:33.071692 13105 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:33.074476 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:33.072308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:39699 (local address 127.30.54.126:35539)
0808 07:11:33.072357 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:33.072359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:33.072616 (+ 257us) server_negotiation.cc:407] Connection header received
0808 07:11:33.073080 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:33.073081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:33.073089 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:33.073140 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:33.074195 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:33.074197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:33.074197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:11:33.074203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:33.074219 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:11:33.074223 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:33.074282 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:33.074387 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 07:11:33.074404 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:11:33.074666 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:33.072304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:35539 (local address 127.30.54.67:39699)
0808 07:11:33.072473 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:33.072522 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:33.072966 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:33.073228 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:33.073231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:33.073238 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:33.074049 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:11:33.074049 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:33.074074 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:11:33.074097 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:33.074295 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:33.074300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:33.074305 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:33.074393 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 07:11:33.074459 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":88,"threads_started":1}
I20260808 07:11:33.075166 13301 heartbeater.cc:347] Connected to a master server at 127.30.54.126:35539
I20260808 07:11:33.075271 13301 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:33.075416 13301 heartbeater.cc:511] Master 127.30.54.126:35539 requested a full tablet report, sending...
I20260808 07:11:33.075795 12636 ts_manager.cc:194] Registered new tserver with Master: d47fc3d833814aed80a5354a397b3f5c (127.30.54.67:35375)
I20260808 07:11:33.080515 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 13105
I20260808 07:11:33.080595 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:33.080744 30937 external_mini_cluster.cc:1428] Setting key db39f6f4c76bfe056ffd96a697169fac
I20260808 07:11:33.081588 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:11:33.083307 12725 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:33.082809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.30.54.126:35539)
0808 07:11:33.082890 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:33.082892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:33.083024 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:11:33.083063 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:33.083064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:33.083068 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:33.083105 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:33.083231 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46564: BlockingRecv error: recv got EOF from 127.0.0.1:46564 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:11:33.083377 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 12728
I20260808 07:11:33.099061 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 12918
I20260808 07:11:33.116405 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 13105
I20260808 07:11:33.128412 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 12584
2026-08-08T07:11:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (575 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:33 dist-test-slave-znh6 krb5kdc[13315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:33 dist-test-slave-znh6 krb5kdc[13315](info): set up 2 sockets
Aug 08 07:11:33 dist-test-slave-znh6 krb5kdc[13315](info): commencing operation
krb5kdc: starting...
W20260808 07:11:34.348577 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:11:34 dist-test-slave-znh6 krb5kdc[13315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:11:34.474654 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:38415
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:38415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:34.555963 13333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:34.556156 13333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:34.556178 13333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:34.557322 13333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:34.557363 13333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:34.557377 13333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:34.557390 13333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:34.557402 13333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:34.558876 13333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:38415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:38415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:34.559124 13333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:34.559363 13333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:34.564654 13344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:34.564728 13333 server_base.cc:1034] running on GCE node
W20260808 07:11:34.564662 13342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:34.564690 13340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:34.565124 13333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:34.565423 13333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:34.566568 13333 hybrid_clock.cc:648] HybridClock initialized: now 1786173094566557 us; error 33 us; skew 500 ppm
Aug 08 07:11:34 dist-test-slave-znh6 krb5kdc[13315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:34.569007 13333 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:11:34.569443 13333 webserver.cc:460] Webserver started at http://127.30.54.126:45461/ using document root <none> and password file <none>
I20260808 07:11:34.569664 13333 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:34.569708 13333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:34.569847 13333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:34.570923 13333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "207b15f17d48412494dd1536b041c9bc"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-znh6"
server_key: "37aa29882dad394a87110461dfd4c9e4"
server_key_iv: "6291477a786f8e34c7afdf3fc39a5b0e"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.571278 13333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "207b15f17d48412494dd1536b041c9bc"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-znh6"
server_key: "37aa29882dad394a87110461dfd4c9e4"
server_key_iv: "6291477a786f8e34c7afdf3fc39a5b0e"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.572597 13333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:34.573323 13359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.573511 13333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:34.573601 13333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "207b15f17d48412494dd1536b041c9bc"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-znh6"
server_key: "37aa29882dad394a87110461dfd4c9e4"
server_key_iv: "6291477a786f8e34c7afdf3fc39a5b0e"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.573675 13333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:34.584071 13333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:34.588431 13333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:34.588590 13333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:34.594095 13333 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:38415
I20260808 07:11:34.594139 13432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:38415 every 8 connection(s)
I20260808 07:11:34.594586 13333 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:34.595675 13433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:34.598021 13433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc: Bootstrap starting.
I20260808 07:11:34.598592 13433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:34.598933 13433 log.cc:824] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:34.599736 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 13333
I20260808 07:11:34.599788 13433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc: No bootstrap required, opened a new log
I20260808 07:11:34.599864 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:11:34.600085 30937 external_mini_cluster.cc:1428] Setting key 1d8003a207871360ad3b2e4bf5fee3ce
I20260808 07:11:34.601758 13433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207b15f17d48412494dd1536b041c9bc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38415 } }
I20260808 07:11:34.601922 13433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:34.601987 13433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 207b15f17d48412494dd1536b041c9bc, State: Initialized, Role: FOLLOWER
I20260808 07:11:34.602118 13433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207b15f17d48412494dd1536b041c9bc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38415 } }
I20260808 07:11:34.602214 13433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:11:34.602270 13433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:11:34.602331 13433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:34.603355 13433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207b15f17d48412494dd1536b041c9bc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38415 } }
I20260808 07:11:34.603528 13433 leader_election.cc:302] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 207b15f17d48412494dd1536b041c9bc; no voters:
I20260808 07:11:34.603729 13433 leader_election.cc:288] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:11:34.603788 13440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:34.604013 13440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [term 1 LEADER]: Becoming Leader. State: Replica: 207b15f17d48412494dd1536b041c9bc, State: Running, Role: LEADER
I20260808 07:11:34.604020 13433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:34.604185 13440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207b15f17d48412494dd1536b041c9bc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38415 } }
I20260808 07:11:34.605849 13440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 207b15f17d48412494dd1536b041c9bc. Latest consensus state: current_term: 1 leader_uuid: "207b15f17d48412494dd1536b041c9bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207b15f17d48412494dd1536b041c9bc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38415 } } }
I20260808 07:11:34.606017 13440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:34.606125 13441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "207b15f17d48412494dd1536b041c9bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207b15f17d48412494dd1536b041c9bc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 38415 } } }
I20260808 07:11:34.606206 13441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc [sys.catalog]: This master's current role is: LEADER
W20260808 07:11:34.606426 13464 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:34.606695 13464 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 07:11:34 dist-test-slave-znh6 krb5kdc[13315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:11:34.611981 13467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:34.612443 13467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:34.613076 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:34.600631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.30.54.126:38415)
0808 07:11:34.600802 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:34.600812 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:34.600855 (+ 43us) server_negotiation.cc:407] Connection header received
0808 07:11:34.601758 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:34.601768 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:34.601855 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:34.602027 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:34.603044 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.604436 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.605675 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.606282 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.606393 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:34.609410 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:34.609414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:34.609414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:34.609427 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:34.611435 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:34.611869 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:34.611873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:34.611873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:34.611988 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:34.612203 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:34.612204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:34.612204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:34.612476 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:34.612623 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:34.612751 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 07:11:34.612892 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260808 07:11:34.622943 13467 catalog_manager.cc:1347] Generated new cluster ID: b4bfc168696947ae989c58f532d96d66
I20260808 07:11:34.622998 13467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:34.630800 13467 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:34.631500 13467 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:34.646064 13467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc: Generated new TSK 0
I20260808 07:11:34.646251 13467 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:11:34.688228 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38415
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:34.808051 13480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:34.808238 13480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:34.808259 13480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:34.809362 13480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:34.809401 13480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:34.809427 13480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:11:34.810889 13480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:34.811117 13480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:34.811357 13480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:34.816330 13493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:34.816308 13489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:34.816466 13480 server_base.cc:1034] running on GCE node
W20260808 07:11:34.816289 13497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:34.816745 13480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:34.816977 13480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:34.818140 13480 hybrid_clock.cc:648] HybridClock initialized: now 1786173094818130 us; error 37 us; skew 500 ppm
Aug 08 07:11:34 dist-test-slave-znh6 krb5kdc[13315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:34.820617 13480 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:11:34.821075 13480 webserver.cc:460] Webserver started at http://127.30.54.65:41615/ using document root <none> and password file <none>
I20260808 07:11:34.821290 13480 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:34.821365 13480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:34.821516 13480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:34.822499 13480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "321a0e60ab4c4ea4a186df5ba3783046"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-znh6"
server_key: "3e7c1ac5ce7a8f193f338ac9855b8421"
server_key_iv: "bff2bc121647428311936c428ae367cf"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.822914 13480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "321a0e60ab4c4ea4a186df5ba3783046"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-znh6"
server_key: "3e7c1ac5ce7a8f193f338ac9855b8421"
server_key_iv: "bff2bc121647428311936c428ae367cf"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.824244 13480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:34.824975 13506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.825145 13480 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:34.825241 13480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "321a0e60ab4c4ea4a186df5ba3783046"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-znh6"
server_key: "3e7c1ac5ce7a8f193f338ac9855b8421"
server_key_iv: "bff2bc121647428311936c428ae367cf"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.825341 13480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:34.841017 13480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:34.843225 13480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:34.843387 13480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:34.843607 13480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:34.843899 13480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:34.843953 13480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.843992 13480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:34.844020 13480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.849864 13480 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:44393
I20260808 07:11:34.849912 13684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:44393 every 8 connection(s)
I20260808 07:11:34.850198 13480 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:11:34.854640 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 13480
I20260808 07:11:34.854768 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:11:34.855028 30937 external_mini_cluster.cc:1428] Setting key 145630efe450a5331519a0e3af71ae0b
Aug 08 07:11:34 dist-test-slave-znh6 krb5kdc[13315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:11:34.859562 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:34.850879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:40715 (local address 127.30.54.126:38415)
0808 07:11:34.850953 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:34.850956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:34.851144 (+ 188us) server_negotiation.cc:407] Connection header received
0808 07:11:34.851668 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:34.851670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:34.851689 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:34.851740 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:34.852630 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.853151 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.854125 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.854345 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.854426 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:34.856700 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:34.856701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:34.856702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:34.856712 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:34.858106 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:34.858565 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:34.858566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:34.858567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:34.858623 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:34.858929 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:34.858930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:34.858931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:34.859043 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:34.859100 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:34.859425 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 07:11:34.859499 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:11:34.859834 13687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:34.850818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:38415 (local address 127.30.54.65:40715)
0808 07:11:34.851025 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:34.851059 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:34.851555 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:34.851810 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:34.851813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:34.852352 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:34.852533 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:34.852537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.853247 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.853248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:34.854032 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:34.854033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:34.854463 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:34.854464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:34.854571 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:34.855127 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:34.855137 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:34.856604 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:34.858212 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:34.858219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:34.858221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:34.858486 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:34.858700 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:34.858701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:34.858701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:34.858869 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:34.859111 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:34.859118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:34.859276 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 07:11:34.859380 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 07:11:34.859557 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260808 07:11:34.860508 13685 heartbeater.cc:347] Connected to a master server at 127.30.54.126:38415
I20260808 07:11:34.860623 13685 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:34.860812 13685 heartbeater.cc:511] Master 127.30.54.126:38415 requested a full tablet report, sending...
I20260808 07:11:34.861456 13386 ts_manager.cc:194] Registered new tserver with Master: 321a0e60ab4c4ea4a186df5ba3783046 (127.30.54.65:44393)
I20260808 07:11:34.862401 13386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:40715
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:11:34.882347 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38415
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:34.961568 13691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:34.961786 13691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:34.961822 13691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:34.962949 13691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:34.963016 13691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:34.963060 13691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:11:34.964512 13691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:34.964772 13691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:34.965000 13691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:34.970112 13691 server_base.cc:1034] running on GCE node
W20260808 07:11:34.970055 13700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:34.970057 13705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:34.970042 13701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:34.970456 13691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:34.970638 13691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:34.971773 13691 hybrid_clock.cc:648] HybridClock initialized: now 1786173094971741 us; error 49 us; skew 500 ppm
Aug 08 07:11:34 dist-test-slave-znh6 krb5kdc[13315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:34.974140 13691 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:11:34.974581 13691 webserver.cc:460] Webserver started at http://127.30.54.66:34831/ using document root <none> and password file <none>
I20260808 07:11:34.974867 13691 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:34.974921 13691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:34.975092 13691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:34.976083 13691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "fead0525e8fb4d2aa7b51153164e3f45"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-znh6"
server_key: "74763da85c8e7ca8384400375e57ecfa"
server_key_iv: "1d16de9fb4910f7f0c2191a70595eee3"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.976428 13691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "fead0525e8fb4d2aa7b51153164e3f45"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-znh6"
server_key: "74763da85c8e7ca8384400375e57ecfa"
server_key_iv: "1d16de9fb4910f7f0c2191a70595eee3"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.977751 13691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:34.978495 13714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.978680 13691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:34.978780 13691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "fead0525e8fb4d2aa7b51153164e3f45"
format_stamp: "Formatted at 2026-08-08 07:11:34 on dist-test-slave-znh6"
server_key: "74763da85c8e7ca8384400375e57ecfa"
server_key_iv: "1d16de9fb4910f7f0c2191a70595eee3"
server_key_version: "encryptionkey@0"
I20260808 07:11:34.978873 13691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:34.987972 13691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:34.989650 13691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:34.989809 13691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:34.990026 13691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:34.990324 13691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:34.990384 13691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.990422 13691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:34.990449 13691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:34.996270 13691 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:42555
I20260808 07:11:34.996320 13880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:42555 every 8 connection(s)
I20260808 07:11:34.996640 13691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:11:34.996845 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 13691
I20260808 07:11:34.996982 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:11:34.997223 30937 external_mini_cluster.cc:1428] Setting key 5e5c178276a45682126e2a1d747dc6d0
Aug 08 07:11:35 dist-test-slave-znh6 krb5kdc[13315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173094, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
I20260808 07:11:35.007071 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:34.997367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:41153 (local address 127.30.54.126:38415)
0808 07:11:34.997418 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:34.997420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:34.997623 (+ 203us) server_negotiation.cc:407] Connection header received
0808 07:11:34.998314 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:34.998315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:34.998325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:34.998376 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:34.999643 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.000170 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.001159 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.001365 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.001456 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.003991 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:35.003992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:35.003993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:35.003999 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:35.005508 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.005995 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.005996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.005997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.006048 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.006330 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.006331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.006331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.006431 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:35.006483 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.006810 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.006955 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:35.007208 13888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:34.997328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:38415 (local address 127.30.54.66:41153)
0808 07:11:34.997490 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:34.997534 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:34.998206 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:34.998445 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:34.998449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:34.999280 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:34.999552 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:34.999558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.000270 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.000271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.001059 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.001061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.001466 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.001467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.001595 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.002090 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:35.002098 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:35.003897 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:35.005602 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:35.005608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:35.005610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:35.005918 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:35.006116 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:35.006117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:35.006117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:35.006256 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:35.006502 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:35.006509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:35.006671 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 07:11:35.006782 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:11:35.006967 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260808 07:11:35.007742 13885 heartbeater.cc:347] Connected to a master server at 127.30.54.126:38415
I20260808 07:11:35.007819 13885 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:35.007946 13885 heartbeater.cc:511] Master 127.30.54.126:38415 requested a full tablet report, sending...
I20260808 07:11:35.008272 13386 ts_manager.cc:194] Registered new tserver with Master: fead0525e8fb4d2aa7b51153164e3f45 (127.30.54.66:42555)
I20260808 07:11:35.008656 13386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:41153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:11:35.025892 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38415
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:35.105863 13893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:35.106050 13893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:35.106073 13893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:35.107306 13893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:35.107347 13893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:35.107374 13893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:11:35.108731 13893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:38415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:11:35.108901 13893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:35.109062 13893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:35.114086 13893 server_base.cc:1034] running on GCE node
W20260808 07:11:35.114019 13904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:35.114055 13901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:35.114022 13900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:35.114439 13893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:35.114629 13893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:35.115769 13893 hybrid_clock.cc:648] HybridClock initialized: now 1786173095115731 us; error 53 us; skew 500 ppm
Aug 08 07:11:35 dist-test-slave-znh6 krb5kdc[13315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:35.118079 13893 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:11:35.118518 13893 webserver.cc:460] Webserver started at http://127.30.54.67:34241/ using document root <none> and password file <none>
I20260808 07:11:35.118753 13893 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:35.118798 13893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:35.119000 13893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:35.120004 13893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "1e67a7f242c2409982bc0f3b2d6f66f6"
format_stamp: "Formatted at 2026-08-08 07:11:35 on dist-test-slave-znh6"
server_key: "f1ab04eda245187bf7201543c4c04609"
server_key_iv: "40d28f89dd0b8a3f490dec8c3275da3b"
server_key_version: "encryptionkey@0"
I20260808 07:11:35.120366 13893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "1e67a7f242c2409982bc0f3b2d6f66f6"
format_stamp: "Formatted at 2026-08-08 07:11:35 on dist-test-slave-znh6"
server_key: "f1ab04eda245187bf7201543c4c04609"
server_key_iv: "40d28f89dd0b8a3f490dec8c3275da3b"
server_key_version: "encryptionkey@0"
I20260808 07:11:35.121675 13893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:35.122357 13913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:35.122545 13893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:11:35.122611 13893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "1e67a7f242c2409982bc0f3b2d6f66f6"
format_stamp: "Formatted at 2026-08-08 07:11:35 on dist-test-slave-znh6"
server_key: "f1ab04eda245187bf7201543c4c04609"
server_key_iv: "40d28f89dd0b8a3f490dec8c3275da3b"
server_key_version: "encryptionkey@0"
I20260808 07:11:35.122701 13893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:35.159018 13893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:35.160799 13893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:35.160959 13893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:35.161196 13893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:11:35.161516 13893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:11:35.161576 13893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:35.161612 13893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:11:35.161638 13893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:35.167675 13893 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:40865
I20260808 07:11:35.167711 14089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:40865 every 8 connection(s)
I20260808 07:11:35.168027 13893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:11:35.170527 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 13893
I20260808 07:11:35.170627 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:11:35.170873 30937 external_mini_cluster.cc:1428] Setting key db812ec7886f3251dd0a3f69eeea6c23
Aug 08 07:11:35 dist-test-slave-znh6 krb5kdc[13315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173095, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:11:35.177338 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.168701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:38167 (local address 127.30.54.126:38415)
0808 07:11:35.168785 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.168788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.169085 (+ 297us) server_negotiation.cc:407] Connection header received
0808 07:11:35.169605 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.169607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.169618 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.169687 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.170737 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.171520 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.172579 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.172768 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.172832 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.174979 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:35.174980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:35.174981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:35.174991 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:35.176248 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.176601 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.176602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.176603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.176652 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.176856 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.176857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.176858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.176954 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:35.177002 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.177217 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.177277 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:11:35.177564 14092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.168658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:38415 (local address 127.30.54.67:38167)
0808 07:11:35.168844 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:35.168999 (+ 155us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:35.169482 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:35.169756 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:35.169760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:35.170364 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:35.170650 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.170656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.171615 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.171615 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.172489 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.172491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.172848 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.172849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.172953 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.173439 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:35.173452 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:35.174888 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:35.176315 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:35.176320 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:35.176322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:35.176536 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:35.176710 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:35.176711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:35.176711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:35.176802 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:35.176996 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:35.177001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:35.177104 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 07:11:35.177187 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:11:35.177338 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260808 07:11:35.178064 14090 heartbeater.cc:347] Connected to a master server at 127.30.54.126:38415
I20260808 07:11:35.178139 14090 heartbeater.cc:464] Registering TS with master...
I20260808 07:11:35.178267 14090 heartbeater.cc:511] Master 127.30.54.126:38415 requested a full tablet report, sending...
I20260808 07:11:35.178584 13386 ts_manager.cc:194] Registered new tserver with Master: 1e67a7f242c2409982bc0f3b2d6f66f6 (127.30.54.67:40865)
I20260808 07:11:35.179035 13386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:38167
I20260808 07:11:35.182520 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:11:35.189867 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.183793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.30.54.126:38415)
0808 07:11:35.183873 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.183876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.183967 (+ 91us) server_negotiation.cc:407] Connection header received
0808 07:11:35.184056 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.184057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.184066 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.184121 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:35.184683 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.185137 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.185846 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.186057 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.186131 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.187332 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:35.187334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:35.187334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:35.187341 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:35.188696 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.189065 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.189066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.189067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.189127 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:35.189327 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:35.189329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:35.189329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:35.189473 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:35.189578 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.189696 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.189772 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:11:35.191617 13386 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:11:35.193364 13386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:11:35.201305 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.196587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:44393 (local address 127.0.0.1:49196)
0808 07:11:35.196708 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:35.196720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:35.196773 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:35.197254 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:35.197257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:35.197259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:35.197322 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.197324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.198706 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.198707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.200062 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.200065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.201017 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.201018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.201109 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.201123 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:11:35.201171 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:11:35.201225 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":100,"thread_start_us":85,"threads_started":1}
I20260808 07:11:35.201534 14107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.196375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:42555 (local address 127.0.0.1:50696)
0808 07:11:35.196506 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:35.196570 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:35.196654 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:35.197292 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:35.197292 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:35.197294 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:35.197332 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.197333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.198659 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.198660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.200061 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.200065 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.201075 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.201075 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.201124 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.201126 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:11:35.201188 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:11:35.201224 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":43,"thread_start_us":54,"threads_started":1}
I20260808 07:11:35.201671 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.196814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.30.54.65:44393)
0808 07:11:35.197062 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.197066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.197076 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:35.197112 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.197114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.197129 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.197418 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:35.197495 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.198540 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.200170 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.200912 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.201437 (+ 525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.201507 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.201532 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.201578 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":102,"threads_started":1}
I20260808 07:11:35.202255 13575 tablet_service.cc:1467] Processing CreateTablet for tablet b0e710db43f441059b86f388b9f15498 (DEFAULT_TABLE table=test-table [id=f6cc9ad245054815abdf5b552c319d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:35.202560 13575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e710db43f441059b86f388b9f15498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:35.202881 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.197202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:40865 (local address 127.0.0.1:33444)
0808 07:11:35.197344 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:35.197354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:35.197401 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:35.197981 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:35.197982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:35.197984 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:35.198046 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.198049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.199591 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.199593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.200995 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.200998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.202266 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.202267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.202363 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.202369 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:11:35.202406 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 07:11:35.202766 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":83,"threads_started":1}
I20260808 07:11:35.203366 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.197543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.30.54.67:40865)
0808 07:11:35.197690 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.197708 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.197718 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:35.197755 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.197758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.197775 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.198130 (+ 355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:35.198184 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.199466 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.201123 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.202146 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.203063 (+ 917us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.203143 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.203173 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.203224 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260808 07:11:35.204218 13995 tablet_service.cc:1467] Processing CreateTablet for tablet b0e710db43f441059b86f388b9f15498 (DEFAULT_TABLE table=test-table [id=f6cc9ad245054815abdf5b552c319d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:35.204622 13995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e710db43f441059b86f388b9f15498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:35.206709 14117 tablet_bootstrap.cc:492] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046: Bootstrap starting.
I20260808 07:11:35.201704 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.196468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.30.54.66:42555)
0808 07:11:35.197053 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.197057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.197070 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:11:35.197113 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.197114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.197130 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.197213 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:35.197412 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.198541 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.200169 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.200923 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.200987 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.201045 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.201268 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.201310 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":88,"threads_started":1}
I20260808 07:11:35.207571 13783 tablet_service.cc:1467] Processing CreateTablet for tablet b0e710db43f441059b86f388b9f15498 (DEFAULT_TABLE table=test-table [id=f6cc9ad245054815abdf5b552c319d5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:11:35.207916 13783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e710db43f441059b86f388b9f15498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:35.207923 14117 tablet_bootstrap.cc:654] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:35.208529 14117 log.cc:824] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:35.209677 14117 tablet_bootstrap.cc:492] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046: No bootstrap required, opened a new log
I20260808 07:11:35.209751 14117 ts_tablet_manager.cc:1397] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:11:35.211057 14121 tablet_bootstrap.cc:492] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: Bootstrap starting.
I20260808 07:11:35.211499 14123 tablet_bootstrap.cc:492] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: Bootstrap starting.
I20260808 07:11:35.211766 14121 tablet_bootstrap.cc:654] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:35.211845 14117 raft_consensus.cc:348] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } }
I20260808 07:11:35.212030 14117 raft_consensus.cc:374] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:35.212074 14117 raft_consensus.cc:729] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 321a0e60ab4c4ea4a186df5ba3783046, State: Initialized, Role: FOLLOWER
I20260808 07:11:35.212170 14123 tablet_bootstrap.cc:654] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:35.212186 14117 consensus_queue.cc:260] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } }
I20260808 07:11:35.212345 14121 log.cc:824] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:35.212479 14117 ts_tablet_manager.cc:1428] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:11:35.212509 13685 heartbeater.cc:503] Master 127.30.54.126:38415 was elected leader, sending a full tablet report...
I20260808 07:11:35.212759 14123 log.cc:824] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:35.213136 14121 tablet_bootstrap.cc:492] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: No bootstrap required, opened a new log
I20260808 07:11:35.213219 14121 ts_tablet_manager.cc:1397] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:35.213732 14123 tablet_bootstrap.cc:492] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: No bootstrap required, opened a new log
I20260808 07:11:35.213801 14123 ts_tablet_manager.cc:1397] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:35.214730 14121 raft_consensus.cc:348] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } }
I20260808 07:11:35.214933 14121 raft_consensus.cc:374] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:35.214998 14121 raft_consensus.cc:729] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e67a7f242c2409982bc0f3b2d6f66f6, State: Initialized, Role: FOLLOWER
I20260808 07:11:35.215111 14121 consensus_queue.cc:260] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } }
I20260808 07:11:35.215390 14090 heartbeater.cc:503] Master 127.30.54.126:38415 was elected leader, sending a full tablet report...
I20260808 07:11:35.215399 14121 ts_tablet_manager.cc:1428] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:35.215297 14123 raft_consensus.cc:348] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } }
I20260808 07:11:35.215478 14123 raft_consensus.cc:374] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:11:35.215521 14123 raft_consensus.cc:729] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fead0525e8fb4d2aa7b51153164e3f45, State: Initialized, Role: FOLLOWER
I20260808 07:11:35.215628 14123 consensus_queue.cc:260] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } }
I20260808 07:11:35.215907 14123 ts_tablet_manager.cc:1428] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:11:35.215931 13885 heartbeater.cc:503] Master 127.30.54.126:38415 was elected leader, sending a full tablet report...
W20260808 07:11:35.247449 13887 tablet.cc:2369] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:35.350965 13686 tablet.cc:2369] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:11:35.418704 14091 tablet.cc:2369] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:11:35.487319 14130 raft_consensus.cc:482] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:11:35.487458 14130 raft_consensus.cc:504] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } }
I20260808 07:11:35.487798 14130 leader_election.cc:288] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fead0525e8fb4d2aa7b51153164e3f45 (127.30.54.66:42555), 321a0e60ab4c4ea4a186df5ba3783046 (127.30.54.65:44393)
I20260808 07:11:35.490736 14092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.488010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:44393 (local address 127.30.54.67:35205)
0808 07:11:35.488080 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:35.488094 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:35.488158 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:35.488374 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:35.488375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:35.488377 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:35.488459 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.488461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.489117 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.489117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.489869 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.489871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.490460 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.490461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.490540 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.490543 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:11:35.490620 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:11:35.490660 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 07:11:35.490736 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.488136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:35205 (local address 127.30.54.65:44393)
0808 07:11:35.488223 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.488226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.488237 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:35.488276 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.488277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.488286 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.488364 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:35.488535 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.489032 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.489983 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.490355 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.490459 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.490468 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.490629 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.490672 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 07:11:35.491104 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e710db43f441059b86f388b9f15498" candidate_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "321a0e60ab4c4ea4a186df5ba3783046" is_pre_election: true
I20260808 07:11:35.491225 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.488143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:60095 (local address 127.30.54.66:42555)
0808 07:11:35.488210 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.488212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.488344 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:11:35.488444 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.488445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.488455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.488532 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:11:35.488691 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.489330 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.490319 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.490692 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.490780 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.490788 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.491120 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.491163 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:11:35.491233 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.488016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:42555 (local address 127.30.54.67:60095)
0808 07:11:35.488281 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:35.488289 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:35.488353 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:35.488536 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:35.488537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:35.488539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:11:35.488620 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.488622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.489479 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.489480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.490224 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:35.490226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.490780 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.490782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:35.491019 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.491024 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:11:35.491113 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 07:11:35.491150 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":82,"thread_start_us":107,"threads_started":1}
I20260808 07:11:35.491272 13614 raft_consensus.cc:2454] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e67a7f242c2409982bc0f3b2d6f66f6 in term 0.
I20260808 07:11:35.491508 13914 leader_election.cc:302] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e67a7f242c2409982bc0f3b2d6f66f6, 321a0e60ab4c4ea4a186df5ba3783046; no voters:
I20260808 07:11:35.491601 13815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e710db43f441059b86f388b9f15498" candidate_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fead0525e8fb4d2aa7b51153164e3f45" is_pre_election: true
I20260808 07:11:35.491664 14130 raft_consensus.cc:2781] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:11:35.491712 14130 raft_consensus.cc:482] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:11:35.491734 14130 raft_consensus.cc:3037] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:35.491755 13815 raft_consensus.cc:2454] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e67a7f242c2409982bc0f3b2d6f66f6 in term 0.
I20260808 07:11:35.492499 14130 raft_consensus.cc:504] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } }
I20260808 07:11:35.492645 14130 leader_election.cc:288] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [CANDIDATE]: Term 1 election: Requested vote from peers fead0525e8fb4d2aa7b51153164e3f45 (127.30.54.66:42555), 321a0e60ab4c4ea4a186df5ba3783046 (127.30.54.65:44393)
I20260808 07:11:35.492832 13815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e710db43f441059b86f388b9f15498" candidate_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fead0525e8fb4d2aa7b51153164e3f45"
I20260808 07:11:35.492853 13614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e710db43f441059b86f388b9f15498" candidate_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "321a0e60ab4c4ea4a186df5ba3783046"
I20260808 07:11:35.492949 13815 raft_consensus.cc:3037] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:35.492942 13614 raft_consensus.cc:3037] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:35.493692 13815 raft_consensus.cc:2454] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e67a7f242c2409982bc0f3b2d6f66f6 in term 1.
I20260808 07:11:35.493716 13614 raft_consensus.cc:2454] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e67a7f242c2409982bc0f3b2d6f66f6 in term 1.
I20260808 07:11:35.493938 13918 leader_election.cc:302] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e67a7f242c2409982bc0f3b2d6f66f6, fead0525e8fb4d2aa7b51153164e3f45; no voters:
I20260808 07:11:35.494064 14130 raft_consensus.cc:2781] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:35.494244 14130 raft_consensus.cc:686] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 1 LEADER]: Becoming Leader. State: Replica: 1e67a7f242c2409982bc0f3b2d6f66f6, State: Running, Role: LEADER
I20260808 07:11:35.494351 14130 consensus_queue.cc:237] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } }
I20260808 07:11:35.495204 13385 catalog_manager.cc:5495] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e67a7f242c2409982bc0f3b2d6f66f6 (127.30.54.67). New cstate: current_term: 1 leader_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e67a7f242c2409982bc0f3b2d6f66f6" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 40865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260808 07:11:35.555787 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:35.553235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50396 (local address 127.30.54.67:40865)
0808 07:11:35.553310 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:35.553312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:35.553323 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:35.553385 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:35.553386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:35.553422 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:35.553507 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:35.553664 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.554135 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.554962 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:35.555165 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:35.555241 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:35.555426 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:35.555535 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:35.555636 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:35.555655 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 07:11:35.555674 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:11:35.557768 13614 raft_consensus.cc:1261] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [term 1 FOLLOWER]: Refusing update from remote peer 1e67a7f242c2409982bc0f3b2d6f66f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:35.557768 13815 raft_consensus.cc:1261] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [term 1 FOLLOWER]: Refusing update from remote peer 1e67a7f242c2409982bc0f3b2d6f66f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:35.558163 14135 consensus_queue.cc:1035] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fead0525e8fb4d2aa7b51153164e3f45" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 42555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:35.558311 14130 consensus_queue.cc:1035] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "321a0e60ab4c4ea4a186df5ba3783046" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:35.560773 14138 mvcc.cc:205] Tried to move back new op lower bound from 7316164999401574400 to 7316164999145320448. Current Snapshot: MvccSnapshot[applied={T|T < 7316164999401574400}]
I20260808 07:11:35.560889 14141 mvcc.cc:205] Tried to move back new op lower bound from 7316164999401574400 to 7316164999145320448. Current Snapshot: MvccSnapshot[applied={T|T < 7316164999401574400}]
I20260808 07:11:35.562911 14140 mvcc.cc:205] Tried to move back new op lower bound from 7316164999401574400 to 7316164999145320448. Current Snapshot: MvccSnapshot[applied={T|T < 7316164999401574400 or (T in {7316164999401574400})}]
I20260808 07:11:35.563066 13385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:35.563176 13385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:11:35.563937 13385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 207b15f17d48412494dd1536b041c9bc: Sending DeleteTablet for 3 replicas of tablet b0e710db43f441059b86f388b9f15498
I20260808 07:11:35.564253 13995 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e710db43f441059b86f388b9f15498 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:35 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:33444
I20260808 07:11:35.564265 13575 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e710db43f441059b86f388b9f15498 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:35 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:49196
I20260808 07:11:35.564567 14155 tablet_replica.cc:330] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046: stopping tablet replica
I20260808 07:11:35.564615 14154 tablet_replica.cc:330] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: stopping tablet replica
I20260808 07:11:35.564604 13783 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e710db43f441059b86f388b9f15498 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:11:35 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:50696
I20260808 07:11:35.564667 14155 raft_consensus.cc:2229] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:35.564702 14154 raft_consensus.cc:2229] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:11:35.564769 14155 raft_consensus.cc:2258] T b0e710db43f441059b86f388b9f15498 P 321a0e60ab4c4ea4a186df5ba3783046 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:35.564803 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 13480
I20260808 07:11:35.564823 14154 raft_consensus.cc:2258] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:35.565683 14154 ts_tablet_manager.cc:1905] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:35.568922 14154 ts_tablet_manager.cc:1918] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:35.569005 14154 log.cc:1196] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/b0e710db43f441059b86f388b9f15498
I20260808 07:11:35.569368 14154 ts_tablet_manager.cc:1939] T b0e710db43f441059b86f388b9f15498 P 1e67a7f242c2409982bc0f3b2d6f66f6: Deleting consensus metadata
I20260808 07:11:35.569814 13363 catalog_manager.cc:4841] TS 1e67a7f242c2409982bc0f3b2d6f66f6 (127.30.54.67:40865): tablet b0e710db43f441059b86f388b9f15498 (table test-table [id=f6cc9ad245054815abdf5b552c319d5f]) successfully deleted
I20260808 07:11:35.589284 14156 tablet_replica.cc:330] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: stopping tablet replica
I20260808 07:11:35.589504 14156 raft_consensus.cc:2229] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:11:35.589685 14156 raft_consensus.cc:2258] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:11:35.591825 14156 ts_tablet_manager.cc:1905] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:11:35.593312 14156 ts_tablet_manager.cc:1918] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:11:35.593367 14156 log.cc:1196] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/b0e710db43f441059b86f388b9f15498
I20260808 07:11:35.593566 14156 ts_tablet_manager.cc:1939] T b0e710db43f441059b86f388b9f15498 P fead0525e8fb4d2aa7b51153164e3f45: Deleting consensus metadata
I20260808 07:11:35.594029 13365 catalog_manager.cc:4841] TS fead0525e8fb4d2aa7b51153164e3f45 (127.30.54.66:42555): tablet b0e710db43f441059b86f388b9f15498 (table test-table [id=f6cc9ad245054815abdf5b552c319d5f]) successfully deleted
W20260808 07:11:35.595988 13360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:44393 (error 108)
W20260808 07:11:35.596529 14115 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:35.596390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:44393 (local address 127.0.0.1:47310)
0808 07:11:35.596455 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:35.596493 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:44393: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 07:11:35.596721 13360 catalog_manager.cc:4584] TS 321a0e60ab4c4ea4a186df5ba3783046 (127.30.54.65:44393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0e710db43f441059b86f388b9f15498: Network error: Client connection negotiation failed: client connection to 127.30.54.65:44393: connect: Connection reset by peer (error 104)
I20260808 07:11:35.597790 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 13691
I20260808 07:11:35.607350 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 13893
I20260808 07:11:35.620213 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 13333
2026-08-08T07:11:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2489 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:35 dist-test-slave-znh6 krb5kdc[14162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:35 dist-test-slave-znh6 krb5kdc[14162](info): set up 2 sockets
Aug 08 07:11:35 dist-test-slave-znh6 krb5kdc[14162](info): commencing operation
krb5kdc: starting...
W20260808 07:11:36.865671 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:11:36 dist-test-slave-znh6 krb5kdc[14162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.54.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.54.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/rangeradmin_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/rangeradmin_127.30.54.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.54.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.54.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/rangerlookup_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/rangerlookup_127.30.54.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.54.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.54.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/HTTP_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/HTTP_127.30.54.84@KRBTEST.COM.keytab.
I20260808 07:11:37.030476 30937 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task8VfA9O/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/postgres -l logfile start
2026-08-08 07:11:39.565 UTC [14203] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:11:39.565 UTC [14203] LOG: listening on IPv4 address "127.30.54.84", port 56917
2026-08-08 07:11:39.569 UTC [14203] LOG: listening on Unix socket "/tmp/.s.PGSQL.56917"
2026-08-08 07:11:39.594 UTC [14208] LOG: database system was shut down at 2026-08-08 07:11:38 UTC
2026-08-08 07:11:39.597 UTC [14203] LOG: database system is ready to accept connections
I20260808 07:11:40.728380 30937 mini_postgres.cc:96] Postgres bound to 56917
2026-08-08 07:11:40.735 UTC [14221] FATAL: database "slave" does not exist
127.30.54.84:56917 - accepting connections
I20260808 07:11:40.736416 30937 mini_ranger.cc:161] Starting Ranger...
I20260808 07:11:40.751273 30937 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:11:41.239372 30937 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:11:41.239446 30937 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-admin
2026-08-08 07:11:41,531 [I] DB FLAVOR :POSTGRES
2026-08-08 07:11:41,531 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:11:41,531 [I] Checking connection..
2026-08-08 07:11:41,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:11:41,791 [I] Checking connection passed.
2026-08-08 07:11:41,791 [I] --------- Verifying version history table ---------
2026-08-08 07:11:41,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:11:42,024 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:11:42,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:11:42,288 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:11:42,288 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:11:42,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:11:42.538 UTC [14321] WARNING: there is no transaction in progress
2026-08-08 07:11:42,545 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:11:42,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:11:42,806 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:11:42,806 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:11:42,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:11:43,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:11:43,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-znh6.c.gcp-upstream.internal','N') ;"
2026-08-08 07:11:43,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:11:43,813 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:11:43,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:11:44,075 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:11:44,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:11:44,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:11:44,583 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:11:44,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:11:45.547 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.563 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.579 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.595 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.611 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.627 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.680 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.688 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.702 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.712 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.722 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.735 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.745 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.751 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.758 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:45.765 UTC [14484] WARNING: there is no transaction in progress
2026-08-08 07:11:46,384 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:11:46,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:11:46,617 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:11:46,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:11:46,880 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:11:46,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:11:47,117 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:11:47,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:11:47,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:11:47,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-znh6.c.gcp-upstream.internal';"
2026-08-08 07:11:47,844 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:11:47,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:11:48,094 [I] DB_PATCHES have already been applied
I20260808 07:11:48.099640 30937 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-admin:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task8VfA9O/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:11:48.099715 30937 mini_ranger.cc:193] Using host: 127.30.54.84
I20260808 07:11:48.103533 30937 mini_ranger.cc:227] Ranger admin URL: http://127.30.54.84:34553
Aug 08, 2026 7:11:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:11:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:11:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:11:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:11:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:11:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:11:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:11:49.442214 30937 mini_ranger.cc:160] Time spent starting Ranger: real 8.706s user 0.003s sys 0.008s
Aug 08, 2026 7:12:09 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:12:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 7:12:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:12:09 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:12:10 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:12:10.705365 30937 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.54.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.54.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/rangerkms_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/rangerkms_127.30.54.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.54.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.54.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.54.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/HTTP_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.54.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/HTTP_127.30.54.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:12:10 dist-test-slave-znh6 krb5kdc[14162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173130, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:12:10.795951 30937 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:12:10.807451 30937 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:12:11.742115 30937 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:12:11.742204 30937 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-kms
I20260808 07:12:11.742225 30937 mini_ranger_kms.cc:227] Using postgres at 127.30.54.84:56917
2026-08-08 07:12:12,184 [I] DB FLAVOR :POSTGRES
2026-08-08 07:12:12,185 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:12:12,185 [I] Checking connection
2026-08-08 07:12:12,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:12:12,482 [I] connection success
2026-08-08 07:12:12,482 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:12:12,482 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:12:12,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:12:12,751 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:12:12,751 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:12:12,752 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:12:12,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:56917/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:12:13,012 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:12:13.491693 30937 mini_ranger_kms.cc:310] Created kms service
I20260808 07:12:13.693251 30937 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:12:13.738152 30937 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:12:13 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173130, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:12:14.137122 30937 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:12:14.137239 30937 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-kms:/tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:12:14.137259 30937 mini_ranger_kms.cc:240] Using host: 127.30.54.84
I20260808 07:12:14.141430 30937 mini_ranger_kms.cc:276] Ranger KMS PID: 14754
I20260808 07:12:14.141559 30937 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.54.84:50745
Aug 08, 2026 7:12:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:12:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:12:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:12:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:12:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:12:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 07:12:16 dist-test-slave-znh6 krb5kdc[14162](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173136, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.54.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:16.834000 30937 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.038s user 0.006s sys 0.413s
I20260808 07:12:16.834127 30937 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:12:16.834230 30937 mini_ranger_kms.cc:401] 127.30.54.84:50745/kms/v1/keys
Aug 08 07:12:17 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173136, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.54.84@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
Aug 08 07:12:17 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173136, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.54.84@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
Aug 08 07:12:19 dist-test-slave-znh6 krb5kdc[14162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:12:19.825816 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:36879
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:36879
--ranger_config_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:50745/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:19.947845 14834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:19.948045 14834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:19.948066 14834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:19.949223 14834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:12:19.949266 14834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:19.949280 14834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:19.949293 14834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:12:19.949304 14834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:12:19.950938 14834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.54.84:50745/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:36879
--ranger_config_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:36879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:12:19.951234 14834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:19.951462 14834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:19.958220 14843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:19.958371 14834 server_base.cc:1034] running on GCE node
W20260808 07:12:19.958233 14841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:19.958416 14845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:19.958787 14834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:19.959133 14834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:19.960342 14834 hybrid_clock.cc:648] HybridClock initialized: now 1786173139960313 us; error 43 us; skew 500 ppm
Aug 08 07:12:19 dist-test-slave-znh6 krb5kdc[14162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173139, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:19.968824 14834 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:12:19.969357 14834 webserver.cc:460] Webserver started at http://127.30.54.126:39715/ using document root <none> and password file <none>
I20260808 07:12:19.969596 14834 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:19.969656 14834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:19.969802 14834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:12:19 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173139, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:12:20.021371 14834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "2a9e5bf6238c48edb1ca103649294a07"
format_stamp: "Formatted at 2026-08-08 07:12:20 on dist-test-slave-znh6"
server_key: "fb88b73c84cf71ed849b5402df1398cc"
server_key_iv: "fb625dcf58902532cfca85328b9043e1"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:20.021833 14834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "2a9e5bf6238c48edb1ca103649294a07"
format_stamp: "Formatted at 2026-08-08 07:12:20 on dist-test-slave-znh6"
server_key: "fb88b73c84cf71ed849b5402df1398cc"
server_key_iv: "fb625dcf58902532cfca85328b9043e1"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:20.023787 14834 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:12:20.038177 14852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:20.038394 14834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:12:20.038482 14834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "2a9e5bf6238c48edb1ca103649294a07"
format_stamp: "Formatted at 2026-08-08 07:12:20 on dist-test-slave-znh6"
server_key: "fb88b73c84cf71ed849b5402df1398cc"
server_key_iv: "fb625dcf58902532cfca85328b9043e1"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:20.038574 14834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:20.064451 14834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:20.066615 14834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:20.066895 14834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:20.072952 14834 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:36879
I20260808 07:12:20.073024 14941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:36879 every 8 connection(s)
I20260808 07:12:20.073465 14834 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:12:20.074272 14942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:20.076854 14942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07: Bootstrap starting.
I20260808 07:12:20.077736 14942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:20.078222 14942 log.cc:824] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:20.078994 14942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07: No bootstrap required, opened a new log
I20260808 07:12:20.081606 14942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9e5bf6238c48edb1ca103649294a07" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36879 } }
I20260808 07:12:20.081779 14942 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:20.081830 14942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a9e5bf6238c48edb1ca103649294a07, State: Initialized, Role: FOLLOWER
I20260808 07:12:20.081977 14942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9e5bf6238c48edb1ca103649294a07" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36879 } }
I20260808 07:12:20.082060 14942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:12:20.082103 14942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:12:20.082151 14942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:20.082716 14942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9e5bf6238c48edb1ca103649294a07" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36879 } }
I20260808 07:12:20.082857 14942 leader_election.cc:302] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a9e5bf6238c48edb1ca103649294a07; no voters:
I20260808 07:12:20.083055 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 14834
I20260808 07:12:20.083061 14942 leader_election.cc:288] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:12:20.083091 14947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:12:20.083168 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:12:20.083274 14947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [term 1 LEADER]: Becoming Leader. State: Replica: 2a9e5bf6238c48edb1ca103649294a07, State: Running, Role: LEADER
I20260808 07:12:20.083362 30937 external_mini_cluster.cc:1428] Setting key d1a29d16aee55bc7aeb17e28f539b2e6
I20260808 07:12:20.083354 14947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9e5bf6238c48edb1ca103649294a07" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36879 } }
I20260808 07:12:20.083415 14942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:12:20.083632 14947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a9e5bf6238c48edb1ca103649294a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9e5bf6238c48edb1ca103649294a07" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36879 } } }
I20260808 07:12:20.083714 14947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [sys.catalog]: This master's current role is: LEADER
I20260808 07:12:20.083874 14947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a9e5bf6238c48edb1ca103649294a07. Latest consensus state: current_term: 1 leader_uuid: "2a9e5bf6238c48edb1ca103649294a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a9e5bf6238c48edb1ca103649294a07" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36879 } } }
I20260808 07:12:20.083941 14947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07 [sys.catalog]: This master's current role is: LEADER
I20260808 07:12:20.085021 14952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:12:20.085573 14952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:12:20.086001 14942 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:12:20.087419 14952 catalog_manager.cc:1347] Generated new cluster ID: d7677023fa2d4211a7c9983a231e9431
I20260808 07:12:20.087477 14952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:12:20 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:12:20.103499 14953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:20.084033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.30.54.126:36879)
0808 07:12:20.084578 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:20.084587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:20.084621 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:12:20.084743 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:20.084751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:20.084823 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:20.087463 (+ 2640us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:20.087664 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:20.089169 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:20.090276 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:20.090894 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:20.091313 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:20.093852 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:20.093856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:20.093856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:20.093868 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:20.100405 (+ 6537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:20.100876 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:20.100879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:20.100880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:20.101038 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:20.101269 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:20.101270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:20.101270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:20.101830 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:20.102459 (+ 629us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:20.102533 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 07:12:20.103018 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":120,"threads_started":1}
I20260808 07:12:20.109138 14952 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:12:20.109875 14952 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:12:20.118700 14952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07: Generated new TSK 0
I20260808 07:12:20.118885 14952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:12:20.979542 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:50745/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36879
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:21.079516 15000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:21.079742 15000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:21.079783 15000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:21.080964 15000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:21.081024 15000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:21.081068 15000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:12:21.082588 15000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.54.84:50745/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:12:21.082870 15000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:21.083112 15000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:21.089645 15014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:21.089774 15011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:21.089991 15016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:21.090883 15000 server_base.cc:1034] running on GCE node
I20260808 07:12:21.091101 15000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:21.091346 15000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:21.092552 15000 hybrid_clock.cc:648] HybridClock initialized: now 1786173141092535 us; error 34 us; skew 500 ppm
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:21.095332 15000 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:12:21.095779 15000 webserver.cc:460] Webserver started at http://127.30.54.65:44149/ using document root <none> and password file <none>
I20260808 07:12:21.096025 15000 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:21.096092 15000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:21.096241 15000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:12:21.112075 15000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "c58a6e76657740b99d614796a8bf8adb"
format_stamp: "Formatted at 2026-08-08 07:12:21 on dist-test-slave-znh6"
server_key: "1bec87c86a5813174a92461d0caae8f6"
server_key_iv: "85419bd407193a5885185f109797bc62"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:21.112514 15000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "c58a6e76657740b99d614796a8bf8adb"
format_stamp: "Formatted at 2026-08-08 07:12:21 on dist-test-slave-znh6"
server_key: "1bec87c86a5813174a92461d0caae8f6"
server_key_iv: "85419bd407193a5885185f109797bc62"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:21.114711 15000 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:12:21.133651 15025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:21.133872 15000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:12:21.133963 15000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "c58a6e76657740b99d614796a8bf8adb"
format_stamp: "Formatted at 2026-08-08 07:12:21 on dist-test-slave-znh6"
server_key: "1bec87c86a5813174a92461d0caae8f6"
server_key_iv: "85419bd407193a5885185f109797bc62"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:21.134055 15000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:21.163482 15000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:21.166193 15000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:21.166411 15000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:21.166769 15000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:21.167299 15000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:21.167349 15000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:21.167389 15000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:21.167447 15000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:21.180840 15000 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:32913
I20260808 07:12:21.180972 15181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:32913 every 8 connection(s)
I20260808 07:12:21.181327 15000 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:12:21.187191 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 15000
I20260808 07:12:21.187305 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:12:21.187534 30937 external_mini_cluster.cc:1428] Setting key 31c6ade24072393d60b86c372680c2dc
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:12:21.196893 15189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.182309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:41349 (local address 127.30.54.126:36879)
0808 07:12:21.182447 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.182450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.182629 (+ 179us) server_negotiation.cc:407] Connection header received
0808 07:12:21.183562 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.183564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.183576 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.183639 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:21.184734 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.185245 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.187019 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.187341 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.187411 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.192756 (+ 5345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:21.192757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:21.192757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:21.192765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:21.194964 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:21.195532 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:21.195533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:21.195534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:21.195604 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:21.195918 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:21.195919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:21.195920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:21.196070 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:21.196126 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.196638 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.196766 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260808 07:12:21.197888 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.182168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36879 (local address 127.30.54.65:41349)
0808 07:12:21.182457 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:21.182517 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:21.183445 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:21.183731 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:21.183736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:21.184158 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:21.184614 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.184621 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.185356 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.185358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.186927 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.186930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.187440 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.187441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.187626 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.188321 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:21.188333 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:21.192643 (+ 4310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:21.195071 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:21.195078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:21.195080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:21.195431 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:21.195683 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:21.195684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:21.195685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:21.195829 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:21.196190 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:21.196205 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:21.196382 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 07:12:21.197360 (+ 978us) client_negotiation.cc:240] Negotiation successful
0808 07:12:21.197582 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":156,"threads_started":1}
I20260808 07:12:21.198757 15182 heartbeater.cc:347] Connected to a master server at 127.30.54.126:36879
I20260808 07:12:21.198984 15182 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:21.199232 15182 heartbeater.cc:511] Master 127.30.54.126:36879 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
I20260808 07:12:21.199867 14879 ts_manager.cc:194] Registered new tserver with Master: c58a6e76657740b99d614796a8bf8adb (127.30.54.65:32913)
I20260808 07:12:21.200670 14879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:41349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:12:21.221294 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:50745/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36879
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:21.339990 15193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:21.340204 15193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:21.340242 15193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:21.341403 15193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:21.341471 15193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:21.341516 15193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:12:21.343321 15193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.54.84:50745/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:12:21.343678 15193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:21.344002 15193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:21.350724 15203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:21.350961 15204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:21.350757 15206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:21.352070 15193 server_base.cc:1034] running on GCE node
I20260808 07:12:21.352268 15193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:21.352538 15193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:21.353693 15193 hybrid_clock.cc:648] HybridClock initialized: now 1786173141353681 us; error 26 us; skew 500 ppm
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:21.356900 15193 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:12:21.357396 15193 webserver.cc:460] Webserver started at http://127.30.54.66:40793/ using document root <none> and password file <none>
I20260808 07:12:21.357687 15193 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:21.357784 15193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:21.357930 15193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:12:21.372720 15193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "628b79012c484a4aa5fa08dd27ef2547"
format_stamp: "Formatted at 2026-08-08 07:12:21 on dist-test-slave-znh6"
server_key: "64a3046e8983a43785b596dd782fe3cc"
server_key_iv: "bd77ee5fa8f3c8298652990fc32b9301"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:21.373211 15193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "628b79012c484a4aa5fa08dd27ef2547"
format_stamp: "Formatted at 2026-08-08 07:12:21 on dist-test-slave-znh6"
server_key: "64a3046e8983a43785b596dd782fe3cc"
server_key_iv: "bd77ee5fa8f3c8298652990fc32b9301"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:21.375350 15193 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:21.388115 15216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:21.388335 15193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:12:21.388430 15193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "628b79012c484a4aa5fa08dd27ef2547"
format_stamp: "Formatted at 2026-08-08 07:12:21 on dist-test-slave-znh6"
server_key: "64a3046e8983a43785b596dd782fe3cc"
server_key_iv: "bd77ee5fa8f3c8298652990fc32b9301"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:21.388520 15193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:21.423220 15193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:21.425910 15193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:21.426117 15193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:21.426496 15193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:21.427016 15193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:21.427114 15193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:21.427189 15193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:21.427246 15193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:21.443781 15193 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:36041
I20260808 07:12:21.444104 15386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:36041 every 8 connection(s)
I20260808 07:12:21.444384 15193 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:12:21.447503 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 15193
I20260808 07:12:21.447614 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:12:21.447799 30937 external_mini_cluster.cc:1428] Setting key 4e892e44a3a98e1daf9fbcf75205c9e6
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
I20260808 07:12:21.460592 15189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.445617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:57823 (local address 127.30.54.126:36879)
0808 07:12:21.445666 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.445668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.445758 (+ 90us) server_negotiation.cc:407] Connection header received
0808 07:12:21.446661 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.446663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.446678 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.446710 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:21.448441 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.449215 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.453203 (+ 3988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.453576 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.453620 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.456751 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:21.456752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:21.456753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:21.456760 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:21.458569 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:21.459214 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:21.459215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:21.459216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:21.459281 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:21.459677 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:21.459678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:21.459679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:21.459851 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:21.459973 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.460390 (+ 417us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.460483 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:12:21.461417 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.445281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36879 (local address 127.30.54.66:57823)
0808 07:12:21.445597 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:21.445648 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:21.446529 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:21.446939 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:21.446944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:21.447312 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:21.448322 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.448329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.451680 (+ 3351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.451681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.453095 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.453098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.453712 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.453712 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.453918 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.454593 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:21.454603 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:21.456598 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:21.458671 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:21.458678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:21.458681 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:21.459105 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:21.459374 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:21.459375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:21.459376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:21.459548 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:21.459995 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:21.460003 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:21.460191 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 07:12:21.460316 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:12:21.460451 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":171,"threads_started":1}
I20260808 07:12:21.462181 15387 heartbeater.cc:347] Connected to a master server at 127.30.54.126:36879
I20260808 07:12:21.462289 15387 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:21.462473 15387 heartbeater.cc:511] Master 127.30.54.126:36879 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
I20260808 07:12:21.462906 14879 ts_manager.cc:194] Registered new tserver with Master: 628b79012c484a4aa5fa08dd27ef2547 (127.30.54.66:36041)
I20260808 07:12:21.463325 14879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:57823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:12:21.482021 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:50745/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36879
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:12:21.594815 15394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:12:21.595223 15394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:12:21.595335 15394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:12:21.596830 15394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:12:21.596894 15394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:12:21.596939 15394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:12:21.598565 15394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.54.84:50745/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:12:21.598811 15394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:12:21.599076 15394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:12:21.606077 15400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:21.606616 15399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:12:21.606905 15402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:12:21.607560 15394 server_base.cc:1034] running on GCE node
I20260808 07:12:21.607774 15394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:12:21.608073 15394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:12:21.609258 15394 hybrid_clock.cc:648] HybridClock initialized: now 1786173141609227 us; error 43 us; skew 500 ppm
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:12:21.612100 15394 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:12:21.612840 15394 webserver.cc:460] Webserver started at http://127.30.54.67:41021/ using document root <none> and password file <none>
I20260808 07:12:21.613117 15394 fs_manager.cc:359] Metadata directory not provided
I20260808 07:12:21.613197 15394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:12:21.613356 15394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:12:21.633788 15394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "a8088b367c8b4a1b90b898fbb3ff56f8"
format_stamp: "Formatted at 2026-08-08 07:12:21 on dist-test-slave-znh6"
server_key: "7fa9f29b37c1196612745b66c748cbc9"
server_key_iv: "64d29e33370dc24478cdd2d38165289d"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:21.634353 15394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "a8088b367c8b4a1b90b898fbb3ff56f8"
format_stamp: "Formatted at 2026-08-08 07:12:21 on dist-test-slave-znh6"
server_key: "7fa9f29b37c1196612745b66c748cbc9"
server_key_iv: "64d29e33370dc24478cdd2d38165289d"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:21.640892 15394 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260808 07:12:21.657392 15415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:21.657672 15394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 07:12:21.657862 15394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "a8088b367c8b4a1b90b898fbb3ff56f8"
format_stamp: "Formatted at 2026-08-08 07:12:21 on dist-test-slave-znh6"
server_key: "7fa9f29b37c1196612745b66c748cbc9"
server_key_iv: "64d29e33370dc24478cdd2d38165289d"
server_key_version: "kuduclusterkey@0"
I20260808 07:12:21.658062 15394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:12:21.675469 15394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:12:21.678311 15394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:12:21.678481 15394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:12:21.678887 15394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:12:21.679423 15394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:12:21.679490 15394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:21.679534 15394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:12:21.679564 15394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:12:21.692190 15394 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:33061
I20260808 07:12:21.692276 15583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:33061 every 8 connection(s)
I20260808 07:12:21.692714 15394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:12:21.696414 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 15394
I20260808 07:12:21.696523 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:12:21.696717 30937 external_mini_cluster.cc:1428] Setting key 5583d8b11deb334c385e714ced62e1e3
Aug 08 07:12:21 dist-test-slave-znh6 krb5kdc[14162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173141, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:12:21.705199 15189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.694053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:45801 (local address 127.30.54.126:36879)
0808 07:12:21.694100 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.694102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.694371 (+ 269us) server_negotiation.cc:407] Connection header received
0808 07:12:21.695442 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.695444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.695457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.695534 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:21.696362 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.697196 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.698442 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.698675 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.698769 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.701234 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:21.701236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:21.701236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:21.701262 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:21.703372 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:21.703916 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:21.703918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:21.703918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:21.704013 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:21.704349 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:21.704351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:21.704351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:21.704555 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:21.704646 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.705046 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.705129 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:12:21.705772 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.693667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36879 (local address 127.30.54.67:45801)
0808 07:12:21.694219 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:21.694264 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:21.695304 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:21.695676 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:21.695681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:21.695895 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:12:21.696195 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.696220 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.697320 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.697321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.698341 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.698344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.698781 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.698782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.698960 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.699556 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:12:21.699570 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:12:21.701118 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:12:21.703477 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:21.703484 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:21.703486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:21.703804 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:21.704119 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:12:21.704120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:12:21.704120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:12:21.704263 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:12:21.704650 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:12:21.704657 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:12:21.704820 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 07:12:21.705355 (+ 535us) client_negotiation.cc:240] Negotiation successful
0808 07:12:21.705519 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":185,"threads_started":1}
I20260808 07:12:21.706354 15584 heartbeater.cc:347] Connected to a master server at 127.30.54.126:36879
I20260808 07:12:21.706458 15584 heartbeater.cc:464] Registering TS with master...
I20260808 07:12:21.706640 15584 heartbeater.cc:511] Master 127.30.54.126:36879 requested a full tablet report, sending...
I20260808 07:12:21.707072 14879 ts_manager.cc:194] Registered new tserver with Master: a8088b367c8b4a1b90b898fbb3ff56f8 (127.30.54.67:33061)
I20260808 07:12:21.707474 14879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:45801
I20260808 07:12:21.708498 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:12:21.717448 15189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.710481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58468 (local address 127.30.54.126:36879)
0808 07:12:21.710520 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.710522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.710718 (+ 196us) server_negotiation.cc:407] Connection header received
0808 07:12:21.710782 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.710782 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.710792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.710880 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:12:21.711454 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.711942 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.712775 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.712971 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.713003 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.714588 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:12:21.714589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:12:21.714590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:12:21.714597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:12:21.716272 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:21.716703 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:21.716704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:21.716704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:21.716760 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:12:21.716970 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:12:21.716971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:12:21.716971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:12:21.717077 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:12:21.717176 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.717279 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.717353 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:12:21.719139 14879 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:12:21.721073 14879 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:12:21.731115 15610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.724230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.30.54.67:33061)
0808 07:12:21.724978 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.724982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.725443 (+ 461us) server_negotiation.cc:407] Connection header received
0808 07:12:21.725599 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.725602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.725618 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.725675 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:21.726013 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.727537 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.729380 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.730289 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.730336 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.730454 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.730913 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.730950 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":150,"threads_started":1}
I20260808 07:12:21.731241 15607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.723742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:33061 (local address 127.0.0.1:52112)
0808 07:12:21.725377 (+ 1635us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:21.725388 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:21.725483 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:21.725845 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:21.725848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:21.725850 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:21.725917 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.725919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.727640 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.727641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.729227 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.729229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.730517 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.730517 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.730593 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.730604 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:12:21.730662 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 07:12:21.730706 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1605,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260808 07:12:21.731992 15486 tablet_service.cc:1467] Processing CreateTablet for tablet f378b4a2532b42f9a56f73a8e8d77f09 (DEFAULT_TABLE table=test-table [id=a2cc2403670e40079880cfde36f39534]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:12:21.732372 15486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f378b4a2532b42f9a56f73a8e8d77f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:21.732599 15606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.723742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:36041 (local address 127.0.0.1:60672)
0808 07:12:21.725068 (+ 1326us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:21.725101 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:21.725195 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:21.727048 (+ 1853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:21.727049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:21.727051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:21.728070 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.728073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.729570 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.729571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.730715 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.730717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.732288 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.732289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.732355 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.732359 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:12:21.732416 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:12:21.732527 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1294,"thread_start_us":126,"threads_started":1}
I20260808 07:12:21.735101 15612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.724468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.30.54.66:36041)
0808 07:12:21.724659 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.724663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.725181 (+ 518us) server_negotiation.cc:407] Connection header received
0808 07:12:21.725336 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.725339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.725357 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.728187 (+ 2830us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:21.728263 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.729463 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.730907 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.732126 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.732985 (+ 859us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.733106 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.734907 (+ 1801us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.734958 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":116,"threads_started":1}
I20260808 07:12:21.735883 15273 tablet_service.cc:1467] Processing CreateTablet for tablet f378b4a2532b42f9a56f73a8e8d77f09 (DEFAULT_TABLE table=test-table [id=a2cc2403670e40079880cfde36f39534]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:12:21.736227 15273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f378b4a2532b42f9a56f73a8e8d77f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:21.739228 15614 tablet_bootstrap.cc:492] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: Bootstrap starting.
I20260808 07:12:21.740521 15614 tablet_bootstrap.cc:654] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:21.741071 15614 log.cc:824] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:21.741392 15615 tablet_bootstrap.cc:492] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: Bootstrap starting.
I20260808 07:12:21.741873 15606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.724053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:32913 (local address 127.0.0.1:42138)
0808 07:12:21.732813 (+ 8760us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:21.732824 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:21.732866 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:21.737632 (+ 4766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:21.737632 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:21.737634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:21.737665 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.737667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.739116 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.739117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.740514 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.740518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.741605 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.741606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.741682 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.741686 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:12:21.741756 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:12:21.741805 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8749,"thread_start_us":71,"threads_started":1}
I20260808 07:12:21.742038 15614 tablet_bootstrap.cc:492] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: No bootstrap required, opened a new log
I20260808 07:12:21.742131 15614 ts_tablet_manager.cc:1397] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:12:21.742128 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.724261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42138 (local address 127.30.54.65:32913)
0808 07:12:21.737460 (+ 13199us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.737465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.737477 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:12:21.737511 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.737513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.737529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.737730 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:21.737790 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.739007 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.740662 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.741411 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.741498 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.741562 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.741797 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.741845 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13161,"thread_start_us":80,"threads_started":1}
I20260808 07:12:21.745785 15614 raft_consensus.cc:348] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } }
I20260808 07:12:21.746027 15614 raft_consensus.cc:374] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:21.746096 15614 raft_consensus.cc:729] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8088b367c8b4a1b90b898fbb3ff56f8, State: Initialized, Role: FOLLOWER
I20260808 07:12:21.746271 15614 consensus_queue.cc:260] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } }
I20260808 07:12:21.747403 15615 tablet_bootstrap.cc:654] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:21.747275 15092 tablet_service.cc:1467] Processing CreateTablet for tablet f378b4a2532b42f9a56f73a8e8d77f09 (DEFAULT_TABLE table=test-table [id=a2cc2403670e40079880cfde36f39534]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:12:21.747699 15092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f378b4a2532b42f9a56f73a8e8d77f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:12:21.748049 15615 log.cc:824] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:21.748463 15614 ts_tablet_manager.cc:1428] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 07:12:21.748611 15584 heartbeater.cc:503] Master 127.30.54.126:36879 was elected leader, sending a full tablet report...
I20260808 07:12:21.749310 15615 tablet_bootstrap.cc:492] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: No bootstrap required, opened a new log
I20260808 07:12:21.749370 15615 ts_tablet_manager.cc:1397] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 07:12:21.751477 15615 raft_consensus.cc:348] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } }
I20260808 07:12:21.751628 15615 raft_consensus.cc:374] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:21.751672 15615 raft_consensus.cc:729] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 628b79012c484a4aa5fa08dd27ef2547, State: Initialized, Role: FOLLOWER
I20260808 07:12:21.751776 15615 consensus_queue.cc:260] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } }
I20260808 07:12:21.751971 15621 tablet_bootstrap.cc:492] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb: Bootstrap starting.
I20260808 07:12:21.752113 15615 ts_tablet_manager.cc:1428] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:12:21.752254 15387 heartbeater.cc:503] Master 127.30.54.126:36879 was elected leader, sending a full tablet report...
I20260808 07:12:21.752933 15621 tablet_bootstrap.cc:654] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb: Neither blocks nor log segments found. Creating new log.
I20260808 07:12:21.753352 15621 log.cc:824] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:12:21.754201 15621 tablet_bootstrap.cc:492] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb: No bootstrap required, opened a new log
I20260808 07:12:21.754273 15621 ts_tablet_manager.cc:1397] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:12:21.757246 15621 raft_consensus.cc:348] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } }
I20260808 07:12:21.757409 15621 raft_consensus.cc:374] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:12:21.757444 15621 raft_consensus.cc:729] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c58a6e76657740b99d614796a8bf8adb, State: Initialized, Role: FOLLOWER
I20260808 07:12:21.757550 15621 consensus_queue.cc:260] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } }
I20260808 07:12:21.757818 15621 ts_tablet_manager.cc:1428] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 07:12:21.757843 15182 heartbeater.cc:503] Master 127.30.54.126:36879 was elected leader, sending a full tablet report...
I20260808 07:12:21.768841 15624 raft_consensus.cc:482] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:12:21.768965 15624 raft_consensus.cc:504] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } }
I20260808 07:12:21.769347 15624 leader_election.cc:288] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8088b367c8b4a1b90b898fbb3ff56f8 (127.30.54.67:33061), 628b79012c484a4aa5fa08dd27ef2547 (127.30.54.66:36041)
I20260808 07:12:21.773154 15612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.769811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:35381 (local address 127.30.54.66:36041)
0808 07:12:21.769854 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.769856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.770074 (+ 218us) server_negotiation.cc:407] Connection header received
0808 07:12:21.770214 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.770215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.770242 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.770283 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:21.770636 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.771201 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.772397 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.772790 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.772857 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.772865 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.773053 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.773088 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:12:21.773234 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.769595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:36041 (local address 127.30.54.65:35381)
0808 07:12:21.769946 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:21.769965 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:21.770087 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:21.770382 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:21.770383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:21.770385 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:21.770546 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.770549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.771303 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.771304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.772305 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.772308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.772863 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.772865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.772965 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.772970 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:12:21.773046 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:12:21.773114 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":20,"thread_start_us":144,"threads_started":1}
I20260808 07:12:21.773597 15321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f378b4a2532b42f9a56f73a8e8d77f09" candidate_uuid: "c58a6e76657740b99d614796a8bf8adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "628b79012c484a4aa5fa08dd27ef2547" is_pre_election: true
I20260808 07:12:21.773732 15610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.769569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:39597 (local address 127.30.54.67:33061)
0808 07:12:21.769694 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.769697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.769762 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:12:21.769869 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.769870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.769879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.769916 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:12:21.770187 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.770749 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.772795 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.773284 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.773390 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.773399 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.773638 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.773676 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 07:12:21.773738 15321 raft_consensus.cc:2454] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c58a6e76657740b99d614796a8bf8adb in term 0.
I20260808 07:12:21.773952 15029 leader_election.cc:302] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 628b79012c484a4aa5fa08dd27ef2547, c58a6e76657740b99d614796a8bf8adb; no voters:
I20260808 07:12:21.773983 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.769494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:33061 (local address 127.30.54.65:39597)
0808 07:12:21.769693 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:21.769705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:12:21.769789 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:12:21.769982 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:12:21.769983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:12:21.769985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:12:21.770122 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.770124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.770900 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.770902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.772686 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:12:21.772690 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.773461 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.773463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:12:21.773544 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.773548 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:12:21.773830 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 07:12:21.773881 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":38,"thread_start_us":108,"threads_started":1}
I20260808 07:12:21.774091 15624 raft_consensus.cc:2781] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:12:21.774137 15624 raft_consensus.cc:482] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:12:21.774165 15624 raft_consensus.cc:3037] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:21.774283 15517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f378b4a2532b42f9a56f73a8e8d77f09" candidate_uuid: "c58a6e76657740b99d614796a8bf8adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" is_pre_election: true
I20260808 07:12:21.774441 15517 raft_consensus.cc:2454] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c58a6e76657740b99d614796a8bf8adb in term 0.
I20260808 07:12:21.775100 15624 raft_consensus.cc:504] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } }
I20260808 07:12:21.775347 15624 leader_election.cc:288] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [CANDIDATE]: Term 1 election: Requested vote from peers a8088b367c8b4a1b90b898fbb3ff56f8 (127.30.54.67:33061), 628b79012c484a4aa5fa08dd27ef2547 (127.30.54.66:36041)
I20260808 07:12:21.775429 15517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f378b4a2532b42f9a56f73a8e8d77f09" candidate_uuid: "c58a6e76657740b99d614796a8bf8adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8"
I20260808 07:12:21.775563 15517 raft_consensus.cc:3037] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:21.777202 15517 raft_consensus.cc:2454] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c58a6e76657740b99d614796a8bf8adb in term 1.
I20260808 07:12:21.777452 15028 leader_election.cc:302] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8088b367c8b4a1b90b898fbb3ff56f8, c58a6e76657740b99d614796a8bf8adb; no voters:
I20260808 07:12:21.777555 15624 raft_consensus.cc:2781] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:12:21.777498 15321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f378b4a2532b42f9a56f73a8e8d77f09" candidate_uuid: "c58a6e76657740b99d614796a8bf8adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "628b79012c484a4aa5fa08dd27ef2547"
I20260808 07:12:21.777710 15624 raft_consensus.cc:686] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 1 LEADER]: Becoming Leader. State: Replica: c58a6e76657740b99d614796a8bf8adb, State: Running, Role: LEADER
I20260808 07:12:21.777681 15321 raft_consensus.cc:3037] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:12:21.777812 15624 consensus_queue.cc:237] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } }
I20260808 07:12:21.778611 14879 catalog_manager.cc:5495] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb reported cstate change: term changed from 0 to 1, leader changed from <none> to c58a6e76657740b99d614796a8bf8adb (127.30.54.65). New cstate: current_term: 1 leader_uuid: "c58a6e76657740b99d614796a8bf8adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c58a6e76657740b99d614796a8bf8adb" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 } health_report { overall_health: UNKNOWN } } }
I20260808 07:12:21.779325 15321 raft_consensus.cc:2454] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c58a6e76657740b99d614796a8bf8adb in term 1.
I20260808 07:12:21.790758 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.787363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.30.54.65:32913)
0808 07:12:21.787402 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:12:21.787404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:12:21.787410 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:12:21.787440 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:12:21.787441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:12:21.787459 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:12:21.787522 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:12:21.787714 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.788428 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.789462 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:12:21.789773 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:12:21.789852 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:12:21.790101 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:12:21.790235 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:12:21.790331 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:12:21.790597 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 07:12:21.790618 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:12:21.793038 15517 raft_consensus.cc:1261] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [term 1 FOLLOWER]: Refusing update from remote peer c58a6e76657740b99d614796a8bf8adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:12:21.793069 15321 raft_consensus.cc:1261] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [term 1 FOLLOWER]: Refusing update from remote peer c58a6e76657740b99d614796a8bf8adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:12:21.793432 15624 consensus_queue.cc:1035] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8088b367c8b4a1b90b898fbb3ff56f8" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:12:21.793599 15624 consensus_queue.cc:1035] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "628b79012c484a4aa5fa08dd27ef2547" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:12:21.798221 15632 mvcc.cc:205] Tried to move back new op lower bound from 7316165188781277184 to 7316165188722348032. Current Snapshot: MvccSnapshot[applied={T|T < 7316165188781277184}]
I20260808 07:12:21.799036 15634 mvcc.cc:205] Tried to move back new op lower bound from 7316165188781277184 to 7316165188722348032. Current Snapshot: MvccSnapshot[applied={T|T < 7316165188781277184}]
I20260808 07:12:21.802014 14879 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:12:21.802145 14879 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:12:21.802958 14879 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a9e5bf6238c48edb1ca103649294a07: Sending DeleteTablet for 3 replicas of tablet f378b4a2532b42f9a56f73a8e8d77f09
I20260808 07:12:21.803599 15486 tablet_service.cc:1514] Processing DeleteTablet for tablet f378b4a2532b42f9a56f73a8e8d77f09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:12:21 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:52112
I20260808 07:12:21.803897 15092 tablet_service.cc:1514] Processing DeleteTablet for tablet f378b4a2532b42f9a56f73a8e8d77f09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:12:21 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:42138
I20260808 07:12:21.804229 15273 tablet_service.cc:1514] Processing DeleteTablet for tablet f378b4a2532b42f9a56f73a8e8d77f09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:12:21 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:60672
I20260808 07:12:21.804864 15643 tablet_replica.cc:330] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: stopping tablet replica
I20260808 07:12:21.804986 15643 raft_consensus.cc:2229] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:12:21.805145 15643 raft_consensus.cc:2258] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:12:21.805361 15644 tablet_replica.cc:330] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb: stopping tablet replica
I20260808 07:12:21.805457 15644 raft_consensus.cc:2229] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:12:21.805577 15644 raft_consensus.cc:2258] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:12:21.805755 15645 tablet_replica.cc:330] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: stopping tablet replica
I20260808 07:12:21.805858 15645 raft_consensus.cc:2229] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:12:21.805924 15645 raft_consensus.cc:2258] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:12:21.806248 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 15000
I20260808 07:12:21.806421 15644 ts_tablet_manager.cc:1905] T f378b4a2532b42f9a56f73a8e8d77f09 P c58a6e76657740b99d614796a8bf8adb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:12:21.807307 15643 ts_tablet_manager.cc:1905] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:12:21.810401 15643 ts_tablet_manager.cc:1918] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:12:21.810485 15643 log.cc:1196] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/f378b4a2532b42f9a56f73a8e8d77f09
I20260808 07:12:21.811213 15643 ts_tablet_manager.cc:1939] T f378b4a2532b42f9a56f73a8e8d77f09 P a8088b367c8b4a1b90b898fbb3ff56f8: Deleting consensus metadata
I20260808 07:12:21.811642 14856 catalog_manager.cc:4841] TS a8088b367c8b4a1b90b898fbb3ff56f8 (127.30.54.67:33061): tablet f378b4a2532b42f9a56f73a8e8d77f09 (table test-table [id=a2cc2403670e40079880cfde36f39534]) successfully deleted
I20260808 07:12:21.815495 15645 ts_tablet_manager.cc:1905] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:12:21.817341 15645 ts_tablet_manager.cc:1918] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:12:21.817430 15645 log.cc:1196] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/f378b4a2532b42f9a56f73a8e8d77f09
I20260808 07:12:21.817824 15645 ts_tablet_manager.cc:1939] T f378b4a2532b42f9a56f73a8e8d77f09 P 628b79012c484a4aa5fa08dd27ef2547: Deleting consensus metadata
I20260808 07:12:21.818269 14857 catalog_manager.cc:4841] TS 628b79012c484a4aa5fa08dd27ef2547 (127.30.54.66:36041): tablet f378b4a2532b42f9a56f73a8e8d77f09 (table test-table [id=a2cc2403670e40079880cfde36f39534]) successfully deleted
W20260808 07:12:21.820330 14856 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:32913 (error 108)
I20260808 07:12:21.821193 15606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:12:21.821046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:32913 (local address 127.0.0.1:42154)
0808 07:12:21.821100 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:12:21.821158 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:32913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 07:12:21.821208 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 15193
W20260808 07:12:21.825435 14856 catalog_manager.cc:4584] TS c58a6e76657740b99d614796a8bf8adb (127.30.54.65:32913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f378b4a2532b42f9a56f73a8e8d77f09: Network error: Client connection negotiation failed: client connection to 127.30.54.65:32913: connect: Connection refused (error 111)
I20260808 07:12:21.834549 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 15394
I20260808 07:12:21.846982 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 14834
I20260808 07:12:21.856514 30937 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:12:22.202255 30937 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:12:22.202359 30937 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:12:22.548426 30937 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:12:22.548 UTC [14203] LOG: received smart shutdown request
2026-08-08 07:12:22.553 UTC [14203] LOG: background worker "logical replication launcher" (PID 14218) exited with exit code 1
2026-08-08 07:12:22.553 UTC [14212] LOG: shutting down
2026-08-08 07:12:22.739 UTC [14203] LOG: database system is shut down
2026-08-08T07:12:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47223 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:12:22 dist-test-slave-znh6 krb5kdc[15663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:12:22 dist-test-slave-znh6 krb5kdc[15663](info): set up 2 sockets
Aug 08 07:12:22 dist-test-slave-znh6 krb5kdc[15663](info): commencing operation
krb5kdc: starting...
W20260808 07:12:24.069429 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:12:24 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:12:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.54.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.54.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/rangeradmin_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/rangeradmin_127.30.54.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.54.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.54.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/rangerlookup_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/rangerlookup_127.30.54.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.54.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.54.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/HTTP_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/HTTP_127.30.54.84@KRBTEST.COM.keytab.
I20260808 07:12:24.236910 30937 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task8VfA9O/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/postgres -l logfile start
2026-08-08 07:12:26.748 UTC [15708] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:12:26.748 UTC [15708] LOG: listening on IPv4 address "127.30.54.84", port 55259
2026-08-08 07:12:26.751 UTC [15708] LOG: listening on Unix socket "/tmp/.s.PGSQL.55259"
2026-08-08 07:12:26.772 UTC [15712] LOG: database system was shut down at 2026-08-08 07:12:25 UTC
2026-08-08 07:12:26.776 UTC [15708] LOG: database system is ready to accept connections
I20260808 07:12:27.892971 30937 mini_postgres.cc:96] Postgres bound to 55259
2026-08-08 07:12:27.900 UTC [15730] FATAL: database "slave" does not exist
127.30.54.84:55259 - accepting connections
I20260808 07:12:27.900624 30937 mini_ranger.cc:161] Starting Ranger...
I20260808 07:12:27.915946 30937 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:12:28.421576 30937 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:12:28.421662 30937 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-admin
2026-08-08 07:12:28,688 [I] DB FLAVOR :POSTGRES
2026-08-08 07:12:28,688 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:12:28,688 [I] Checking connection..
2026-08-08 07:12:28,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:12:28,947 [I] Checking connection passed.
2026-08-08 07:12:28,947 [I] --------- Verifying version history table ---------
2026-08-08 07:12:28,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:12:29,198 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:12:29,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:12:29,465 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:12:29,465 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:12:29,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:12:29.730 UTC [15825] WARNING: there is no transaction in progress
2026-08-08 07:12:29,737 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:12:29,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:12:30,000 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:12:30,001 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:12:30,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:12:30,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:12:30,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-znh6.c.gcp-upstream.internal','N') ;"
2026-08-08 07:12:30,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:12:31,009 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:12:31,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:12:31,250 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:12:31,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:12:31,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:12:31,760 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:12:31,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:12:32.680 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.696 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.712 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.727 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.742 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.759 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.816 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.822 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.836 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.845 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.856 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.869 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.878 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.885 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.891 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:32.898 UTC [15983] WARNING: there is no transaction in progress
2026-08-08 07:12:33,552 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:12:33,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:12:33,803 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:12:33,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:12:34,045 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:12:34,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:12:34,278 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:12:34,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:12:34,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:12:34,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-znh6.c.gcp-upstream.internal';"
2026-08-08 07:12:35,033 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:12:35,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:12:35,297 [I] DB_PATCHES have already been applied
I20260808 07:12:35.303102 30937 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-admin:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task8VfA9O/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:12:35.303164 30937 mini_ranger.cc:193] Using host: 127.30.54.84
I20260808 07:12:35.307108 30937 mini_ranger.cc:227] Ranger admin URL: http://127.30.54.84:50973
Aug 08, 2026 7:12:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:12:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:12:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:12:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:12:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:12:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:12:36 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:12:36.655309 30937 mini_ranger.cc:160] Time spent starting Ranger: real 8.755s user 0.000s sys 0.010s
Aug 08, 2026 7:12:56 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:12:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 7:12:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:12:57 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:12:57 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:12:58.480086 30937 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.54.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.54.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/rangerkms_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.54.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/rangerkms_127.30.54.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.54.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.54.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.54.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/HTTP_127.30.54.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.54.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/HTTP_127.30.54.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:12:58 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173178, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:12:58.560369 30937 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:12:58.570469 30937 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:12:59.422648 30937 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:12:59.422740 30937 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-kms
I20260808 07:12:59.422762 30937 mini_ranger_kms.cc:227] Using postgres at 127.30.54.84:55259
2026-08-08 07:12:59,597 [I] DB FLAVOR :POSTGRES
2026-08-08 07:12:59,597 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:12:59,597 [I] Checking connection
2026-08-08 07:12:59,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:12:59,864 [I] connection success
2026-08-08 07:12:59,865 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:12:59,865 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:12:59,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:13:00,096 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:13:00,096 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:13:00,096 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:13:00,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.54.84:55259/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:13:00,378 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:13:00.858956 30937 mini_ranger_kms.cc:310] Created kms service
I20260808 07:13:01.064348 30937 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:13:01.106154 30937 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:13:01 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173178, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:01.498637 30937 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:13:01.499038 30937 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-kms:/tmp/dist-test-task8VfA9O/build/release/bin/postgresql.jar:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task8VfA9O/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:13:01.499089 30937 mini_ranger_kms.cc:240] Using host: 127.30.54.84
I20260808 07:13:01.503077 30937 mini_ranger_kms.cc:276] Ranger KMS PID: 16259
I20260808 07:13:01.503153 30937 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.54.84:53949
Aug 08, 2026 7:13:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:13:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:13:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:13:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:13:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:13:03 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 07:13:03 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173183, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.54.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:04.248426 30937 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.688s user 0.003s sys 0.155s
I20260808 07:13:04.248524 30937 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:13:04.248572 30937 mini_ranger_kms.cc:401] 127.30.54.84:53949/kms/v1/keys
Aug 08 07:13:04 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173183, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.54.84@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
Aug 08 07:13:04 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173183, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.54.84@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
Aug 08 07:13:06 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:13:06.949620 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:33209
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:33209
--ranger_config_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:53949/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:07.030121 16331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:07.030310 16331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:07.030332 16331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:07.031508 16331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:07.031550 16331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:07.031565 16331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:07.031579 16331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:07.031591 16331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:07.033169 16331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.54.84:53949/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:33209
--ranger_config_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:33209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:07.033412 16331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:07.033636 16331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:07.038928 16343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:07.038960 16340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:07.038966 16338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:07.038960 16331 server_base.cc:1034] running on GCE node
I20260808 07:13:07.039356 16331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:07.039589 16331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:07.040727 16331 hybrid_clock.cc:648] HybridClock initialized: now 1786173187040718 us; error 33 us; skew 500 ppm
Aug 08 07:13:07 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173187, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:07.043345 16331 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:13:07.043815 16331 webserver.cc:460] Webserver started at http://127.30.54.126:33997/ using document root <none> and password file <none>
I20260808 07:13:07.044054 16331 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:07.044095 16331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:07.044236 16331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:13:07 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173187, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:07.098362 16331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc"
format_stamp: "Formatted at 2026-08-08 07:13:07 on dist-test-slave-znh6"
server_key: "52300cae204bd20039c781f134a20321"
server_key_iv: "57d99ef373811b22fa61a6aa8fd6b36d"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:07.099097 16331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc"
format_stamp: "Formatted at 2026-08-08 07:13:07 on dist-test-slave-znh6"
server_key: "52300cae204bd20039c781f134a20321"
server_key_iv: "57d99ef373811b22fa61a6aa8fd6b36d"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:07.101009 16331 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 07:13:07.115293 16356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:07.115516 16331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:07.115626 16331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc"
format_stamp: "Formatted at 2026-08-08 07:13:07 on dist-test-slave-znh6"
server_key: "52300cae204bd20039c781f134a20321"
server_key_iv: "57d99ef373811b22fa61a6aa8fd6b36d"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:07.115703 16331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:07.133162 16331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:07.135196 16331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:07.135603 16331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:07.140568 16331 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:33209
I20260808 07:13:07.140604 16431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:33209 every 8 connection(s)
I20260808 07:13:07.141065 16331 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:07.141800 16432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:07.144150 16432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Bootstrap starting.
I20260808 07:13:07.144858 16432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:07.145167 16432 log.cc:824] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:07.145429 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 16331
I20260808 07:13:07.145531 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:13:07.145709 30937 external_mini_cluster.cc:1428] Setting key 781a26840a61f82a13edabdb1e88290b
I20260808 07:13:07.146030 16432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: No bootstrap required, opened a new log
I20260808 07:13:07.149140 16432 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } }
I20260808 07:13:07.149303 16432 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:07.149333 16432 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea4bf5cf3eaf4728a63e853f3c0757fc, State: Initialized, Role: FOLLOWER
I20260808 07:13:07.149513 16432 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } }
I20260808 07:13:07.149621 16432 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:07.149680 16432 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:07.149739 16432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:07.150281 16432 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } }
I20260808 07:13:07.150403 16432 leader_election.cc:302] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea4bf5cf3eaf4728a63e853f3c0757fc; no voters:
I20260808 07:13:07.150578 16432 leader_election.cc:288] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:13:07.150612 16438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:07.150833 16438 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 1 LEADER]: Becoming Leader. State: Replica: ea4bf5cf3eaf4728a63e853f3c0757fc, State: Running, Role: LEADER
I20260808 07:13:07.150969 16438 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } }
I20260808 07:13:07.150982 16432 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:07.152900 16432 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:13:07.151314 16438 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } } }
I20260808 07:13:07.153369 16438 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: This master's current role is: LEADER
Aug 08 07:13:07 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:07.155484 16438 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea4bf5cf3eaf4728a63e853f3c0757fc. Latest consensus state: current_term: 1 leader_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } } }
I20260808 07:13:07.155648 16438 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:07.157400 16463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:07.158293 16463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:07.160564 16463 catalog_manager.cc:1347] Generated new cluster ID: 581c386a28474d9893a22c84d33bf87b
I20260808 07:13:07.160667 16463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:13:07.161360 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:07.146242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.30.54.126:33209)
0808 07:13:07.146410 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:07.146420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:07.146459 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:13:07.146994 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:07.147004 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:07.147064 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:07.147214 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:07.147974 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:07.149392 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:07.150485 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:07.151157 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:07.151271 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:07.156120 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:07.156173 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:07.156173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:07.156263 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:07.159286 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:07.159699 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:07.159701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:07.159702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:07.159798 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:07.160014 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:07.160015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:07.160015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:07.160340 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:07.160487 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:07.161014 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 07:13:07.161157 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260808 07:13:07.169433 16463 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:13:07.170115 16463 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:07.172335 16463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Generated new TSK 0
I20260808 07:13:07.172508 16463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:13:07.849941 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:53949/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:33209
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:07.939257 16499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:07.939445 16499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:07.939467 16499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:07.940606 16499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:07.940649 16499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:07.940673 16499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:13:07.942135 16499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.54.84:53949/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:07.942345 16499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:07.942611 16499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:07.948688 16506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:07.948803 16511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:07.948920 16499 server_base.cc:1034] running on GCE node
W20260808 07:13:07.948701 16507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:07.949190 16499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:07.949522 16499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:07.950750 16499 hybrid_clock.cc:648] HybridClock initialized: now 1786173187950728 us; error 36 us; skew 500 ppm
Aug 08 07:13:07 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173187, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:07.953392 16499 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:13:07.953989 16499 webserver.cc:460] Webserver started at http://127.30.54.65:45861/ using document root <none> and password file <none>
I20260808 07:13:07.954303 16499 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:07.954411 16499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:07.954566 16499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:13:07 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173187, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:07.968398 16499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "b40059ead5b649b285b663539c009c5a"
format_stamp: "Formatted at 2026-08-08 07:13:07 on dist-test-slave-znh6"
server_key: "7b502965e27d11c423bb624390b2fe1b"
server_key_iv: "8eb5b506be8b4d5a51f7ae9da7586d4a"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:07.968918 16499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "b40059ead5b649b285b663539c009c5a"
format_stamp: "Formatted at 2026-08-08 07:13:07 on dist-test-slave-znh6"
server_key: "7b502965e27d11c423bb624390b2fe1b"
server_key_iv: "8eb5b506be8b4d5a51f7ae9da7586d4a"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:07.970518 16499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:07.983218 16526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:07.983420 16499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:07.983505 16499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "b40059ead5b649b285b663539c009c5a"
format_stamp: "Formatted at 2026-08-08 07:13:07 on dist-test-slave-znh6"
server_key: "7b502965e27d11c423bb624390b2fe1b"
server_key_iv: "8eb5b506be8b4d5a51f7ae9da7586d4a"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:07.983587 16499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:08.001273 16499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:08.003700 16499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:08.003912 16499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:08.004199 16499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:08.004587 16499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:08.004666 16499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.004724 16499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:08.004777 16499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.012292 16499 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:37697
I20260808 07:13:08.012745 16499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:08.013226 16694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:37697 every 8 connection(s)
I20260808 07:13:08.016532 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 16499
I20260808 07:13:08.016637 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:13:08.016880 30937 external_mini_cluster.cc:1428] Setting key 517a034fc8573bee09914869ba98d431
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173187, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:08.025664 16702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:08.013660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:52587 (local address 127.30.54.126:33209)
0808 07:13:08.015900 (+ 2240us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:08.015902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:08.015912 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:08.015940 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:08.015941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:08.015955 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:08.016031 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:08.016794 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.017715 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.019264 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.019616 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.019681 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:08.022303 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:08.022305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:08.022305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:08.022312 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:08.024169 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:08.024624 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:08.024625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:08.024626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:08.024684 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:08.024944 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:08.024945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:08.024945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:08.025075 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:08.025149 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:08.025472 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 07:13:08.025568 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2205,"thread_start_us":85,"threads_started":1}
I20260808 07:13:08.025844 16701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:08.013644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:33209 (local address 127.30.54.65:52587)
0808 07:13:08.014935 (+ 1291us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:08.014986 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:08.015699 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:08.016148 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:08.016153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:08.016389 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:08.016655 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:08.016661 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.017838 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.017839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:08.019160 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:08.019163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.019764 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.019765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:08.019908 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:08.020485 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:08.020495 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:08.022195 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:08.024263 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:08.024270 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:08.024272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:08.024543 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:08.024747 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:08.024748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:08.024748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:08.024863 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:08.025159 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:08.025166 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:08.025318 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 07:13:08.025446 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 07:13:08.025589 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1245,"thread_start_us":83,"threads_started":1}
I20260808 07:13:08.026432 16695 heartbeater.cc:347] Connected to a master server at 127.30.54.126:33209
I20260808 07:13:08.026571 16695 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:08.026790 16695 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
I20260808 07:13:08.027423 16377 ts_manager.cc:194] Registered new tserver with Master: b40059ead5b649b285b663539c009c5a (127.30.54.65:37697)
I20260808 07:13:08.028303 16377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:52587
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:13:08.057243 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:53949/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:33209
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:08.179176 16709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:08.179404 16709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:08.179446 16709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:08.180557 16709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:08.180624 16709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:08.180668 16709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:13:08.182055 16709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.54.84:53949/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:08.182314 16709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:08.182559 16709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:08.187558 16730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:08.187559 16726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:08.187584 16732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:08.188500 16709 server_base.cc:1034] running on GCE node
I20260808 07:13:08.188704 16709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:08.188964 16709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:08.190119 16709 hybrid_clock.cc:648] HybridClock initialized: now 1786173188190114 us; error 34 us; skew 500 ppm
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:08.192740 16709 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:13:08.193186 16709 webserver.cc:460] Webserver started at http://127.30.54.66:40645/ using document root <none> and password file <none>
I20260808 07:13:08.193421 16709 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:08.193492 16709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:08.193634 16709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:08.211006 16709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "dfc765f0ed594e6a8ad1e3941472b16e"
format_stamp: "Formatted at 2026-08-08 07:13:08 on dist-test-slave-znh6"
server_key: "e93cfaa825ec17d59770dea13e0cf097"
server_key_iv: "11ff40c461bdb0adad3092f6ef0ec07f"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:08.211488 16709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "dfc765f0ed594e6a8ad1e3941472b16e"
format_stamp: "Formatted at 2026-08-08 07:13:08 on dist-test-slave-znh6"
server_key: "e93cfaa825ec17d59770dea13e0cf097"
server_key_iv: "11ff40c461bdb0adad3092f6ef0ec07f"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:08.215128 16709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 07:13:08.230959 16740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.231132 16709 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260808 07:13:08.231220 16709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "dfc765f0ed594e6a8ad1e3941472b16e"
format_stamp: "Formatted at 2026-08-08 07:13:08 on dist-test-slave-znh6"
server_key: "e93cfaa825ec17d59770dea13e0cf097"
server_key_iv: "11ff40c461bdb0adad3092f6ef0ec07f"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:08.231320 16709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:08.248417 16709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:08.250770 16709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:08.250945 16709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:08.251170 16709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:08.251636 16709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:08.251688 16709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.251726 16709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:08.251808 16709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.258800 16709 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:39199
I20260808 07:13:08.259564 16709 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:13:08.261104 16911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:39199 every 8 connection(s)
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:08.268651 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 16709
I20260808 07:13:08.268745 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:13:08.268944 30937 external_mini_cluster.cc:1428] Setting key c316d0820fc63dffbd5af48b1426dabd
I20260808 07:13:08.273391 16915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:08.261735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:33209 (local address 127.30.54.66:60647)
0808 07:13:08.261958 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:08.262003 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:08.262560 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:08.262864 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:08.262869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:08.263118 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:08.263407 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:08.263413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.264466 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.264467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:08.265696 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:08.265699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.266282 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.266283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:08.266421 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:08.267055 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:08.267064 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:08.268766 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:08.271782 (+ 3016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:08.271789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:08.271791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:08.272085 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:08.272326 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:08.272327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:08.272327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:08.272462 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:08.272751 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:08.272756 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:08.272916 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 07:13:08.273014 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:13:08.273166 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260808 07:13:08.274048 16702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:08.262074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:60647 (local address 127.30.54.126:33209)
0808 07:13:08.262117 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:08.262120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:08.262129 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:08.262681 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:08.262682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:08.262700 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:08.262752 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:08.263529 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.264342 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.265820 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.266118 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.266169 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:08.268889 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:08.268891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:08.268891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:08.268899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:08.270782 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:08.272163 (+ 1381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:08.272164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:08.272164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:08.272234 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:08.272535 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:08.272535 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:08.272536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:08.272669 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:08.272726 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:08.273857 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0808 07:13:08.273952 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:13:08.274397 16912 heartbeater.cc:347] Connected to a master server at 127.30.54.126:33209
I20260808 07:13:08.274507 16912 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:08.274703 16912 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
I20260808 07:13:08.275098 16377 ts_manager.cc:194] Registered new tserver with Master: dfc765f0ed594e6a8ad1e3941472b16e (127.30.54.66:39199)
I20260808 07:13:08.275589 16377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:60647
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:13:08.301383 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:53949/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:33209
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:08.420006 16920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:08.420269 16920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:08.420305 16920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:08.421989 16920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:08.422043 16920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:08.422080 16920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:13:08.424238 16920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.54.84:53949/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:08.424475 16920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:08.424763 16920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:08.432484 16936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:08.432760 16931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:08.433027 16932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:08.433301 16920 server_base.cc:1034] running on GCE node
I20260808 07:13:08.433497 16920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:08.433753 16920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:08.434895 16920 hybrid_clock.cc:648] HybridClock initialized: now 1786173188434874 us; error 37 us; skew 500 ppm
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:08.438603 16920 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:13:08.439201 16920 webserver.cc:460] Webserver started at http://127.30.54.67:43813/ using document root <none> and password file <none>
I20260808 07:13:08.439512 16920 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:08.439607 16920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:08.439785 16920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:08.461474 16920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "b2d6ac491e1b4f85b88969a62b30560e"
format_stamp: "Formatted at 2026-08-08 07:13:08 on dist-test-slave-znh6"
server_key: "11f605cd2a2ecb34532901b5c78b3e38"
server_key_iv: "d3151a279bd2075266d1853f00f8bb01"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:08.462071 16920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "b2d6ac491e1b4f85b88969a62b30560e"
format_stamp: "Formatted at 2026-08-08 07:13:08 on dist-test-slave-znh6"
server_key: "11f605cd2a2ecb34532901b5c78b3e38"
server_key_iv: "d3151a279bd2075266d1853f00f8bb01"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:08.464159 16920 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 07:13:08.494199 16950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.494459 16920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:13:08.494552 16920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "b2d6ac491e1b4f85b88969a62b30560e"
format_stamp: "Formatted at 2026-08-08 07:13:08 on dist-test-slave-znh6"
server_key: "11f605cd2a2ecb34532901b5c78b3e38"
server_key_iv: "d3151a279bd2075266d1853f00f8bb01"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:08.494637 16920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:08.506213 16920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:08.508381 16920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:08.508529 16920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:08.508771 16920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:08.509171 16920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:08.509208 16920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.509231 16920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:08.509271 16920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.519482 16920 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:34743
I20260808 07:13:08.519630 17135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:34743 every 8 connection(s)
I20260808 07:13:08.519901 16920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:08.528396 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 16920
I20260808 07:13:08.528512 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:13:08.528707 30937 external_mini_cluster.cc:1428] Setting key 3bdc2fe70004e11e79032b9feda11412
I20260808 07:13:08.530516 16702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:08.520723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:34257 (local address 127.30.54.126:33209)
0808 07:13:08.520794 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:08.520796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:08.521001 (+ 205us) server_negotiation.cc:407] Connection header received
0808 07:13:08.521564 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:08.521566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:08.521579 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:08.521649 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:08.522258 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.523068 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.524048 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.524356 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.524438 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:08.526524 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:08.526525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:08.526526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:08.526535 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:08.528511 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:08.529145 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:08.529146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:08.529147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:08.529214 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:08.529720 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:08.529721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:08.529722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:08.529843 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:08.529893 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:08.530388 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 07:13:08.530454 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:13:08.531039 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:08.520683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:33209 (local address 127.30.54.67:34257)
0808 07:13:08.520868 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:08.520909 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:08.521458 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:08.521737 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:08.521742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:08.521943 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:08.522161 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:08.522165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.523158 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.523159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:08.523957 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:08.523959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:08.524458 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:08.524459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:08.524564 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:08.525099 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:08.525107 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:08.526423 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:08.528635 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:08.528643 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:08.528646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:08.529012 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:08.529294 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:08.529302 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:08.529303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:08.529627 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:08.529924 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:08.529931 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:08.530197 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 07:13:08.530384 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 07:13:08.530631 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
I20260808 07:13:08.531720 17137 heartbeater.cc:347] Connected to a master server at 127.30.54.126:33209
I20260808 07:13:08.531903 17137 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:08.532223 17137 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
I20260808 07:13:08.532694 16377 ts_manager.cc:194] Registered new tserver with Master: b2d6ac491e1b4f85b88969a62b30560e (127.30.54.67:34743)
I20260808 07:13:08.533068 16377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:34257
I20260808 07:13:08.540406 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:08.541249 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 16499
I20260808 07:13:08.566020 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 16709
I20260808 07:13:08.575057 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 16920
I20260808 07:13:08.595120 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 16331
I20260808 07:13:08.605448 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:33209
--webserver_interface=127.30.54.126
--webserver_port=33997
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:33209
--ranger_config_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:53949/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.54.84:53949/kms with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:08.687852 17141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:08.688050 17141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:08.688071 17141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:08.689143 17141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:08.689178 17141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:08.689193 17141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:08.689204 17141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:08.689216 17141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:08.690688 17141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.54.84:53949/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:33209
--ranger_config_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:33209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=33997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:08.690927 17141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:08.691100 17141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:08.695758 17152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:08.695776 17149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:08.695812 17141 server_base.cc:1034] running on GCE node
W20260808 07:13:08.695780 17156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:08.696242 17141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:08.696538 17141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:08.697705 17141 hybrid_clock.cc:648] HybridClock initialized: now 1786173188697684 us; error 36 us; skew 500 ppm
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:08.700111 17141 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:13:08.700630 17141 webserver.cc:460] Webserver started at http://127.30.54.126:33997/ using document root <none> and password file <none>
I20260808 07:13:08.700851 17141 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:08.700927 17141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:08.777310 17141 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:08.778133 17167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.778322 17141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:08.778434 17141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc"
format_stamp: "Formatted at 2026-08-08 07:13:07 on dist-test-slave-znh6"
server_key: "52300cae204bd20039c781f134a20321"
server_key_iv: "57d99ef373811b22fa61a6aa8fd6b36d"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:08.778939 17141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:08.795990 17141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:08.798125 17141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:08.798288 17141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:08.803664 17141 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:33209
I20260808 07:13:08.803689 17251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:33209 every 8 connection(s)
I20260808 07:13:08.804165 17141 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:08.805415 17252 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:13:08.806052 17252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Bootstrap starting.
I20260808 07:13:08.811978 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 17141
I20260808 07:13:08.812325 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:53949/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:37697
--local_ip_for_outbound_sockets=127.30.54.65
--tserver_master_addrs=127.30.54.126:33209
--webserver_port=45861
--webserver_interface=127.30.54.65
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.54.84:53949/kms with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:08.814488 17252 log.cc:824] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:08.816207 17252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:08.816546 17252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Bootstrap complete.
I20260808 07:13:08.820595 17252 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } }
I20260808 07:13:08.820979 17252 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea4bf5cf3eaf4728a63e853f3c0757fc, State: Initialized, Role: FOLLOWER
I20260808 07:13:08.821197 17252 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } }
I20260808 07:13:08.821290 17252 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:08.821344 17252 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:08.821393 17252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:13:08.822992 17252 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } }
I20260808 07:13:08.823117 17252 leader_election.cc:302] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea4bf5cf3eaf4728a63e853f3c0757fc; no voters:
I20260808 07:13:08.823298 17252 leader_election.cc:288] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:13:08.823436 17256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:13:08.823654 17256 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [term 2 LEADER]: Becoming Leader. State: Replica: ea4bf5cf3eaf4728a63e853f3c0757fc, State: Running, Role: LEADER
I20260808 07:13:08.823863 17256 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } }
I20260808 07:13:08.823657 17252 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:08.824244 17257 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } } }
I20260808 07:13:08.824337 17257 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:08.824509 17259 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea4bf5cf3eaf4728a63e853f3c0757fc. Latest consensus state: current_term: 2 leader_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea4bf5cf3eaf4728a63e853f3c0757fc" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 33209 } } }
I20260808 07:13:08.824585 17259 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:08.825119 17261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:08.825642 17261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:08.825791 17261 catalog_manager.cc:1259] Loaded cluster ID: 581c386a28474d9893a22c84d33bf87b
I20260808 07:13:08.825837 17261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:13:08.826200 17252 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:13:08.827157 17261 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:08.827332 17261 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Loaded TSK: 0
I20260808 07:13:08.827540 17261 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:13:08.930419 17254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:08.930691 17254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:08.930722 17254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:08.932281 17254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:08.932335 17254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:08.932371 17254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:13:08.934283 17254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.54.84:53949/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:37697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=45861
--tserver_master_addrs=127.30.54.126:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:08.934517 17254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:08.934767 17254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:08.941315 17306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:08.941635 17303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:08.941804 17308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:08.942884 17254 server_base.cc:1034] running on GCE node
I20260808 07:13:08.943157 17254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:08.943424 17254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:08.946884 17254 hybrid_clock.cc:648] HybridClock initialized: now 1786173188946846 us; error 50 us; skew 500 ppm
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:08.953064 17254 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:13:08.953552 17254 webserver.cc:460] Webserver started at http://127.30.54.65:45861/ using document root <none> and password file <none>
I20260808 07:13:08.953785 17254 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:08.953850 17254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:13:08 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:08.972740 17254 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 07:13:08.973779 17317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:08.973999 17254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:08.974080 17254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "b40059ead5b649b285b663539c009c5a"
format_stamp: "Formatted at 2026-08-08 07:13:07 on dist-test-slave-znh6"
server_key: "7b502965e27d11c423bb624390b2fe1b"
server_key_iv: "8eb5b506be8b4d5a51f7ae9da7586d4a"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:08.974507 17254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:09.004228 17254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:09.006662 17254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:09.006894 17254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:09.007143 17254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:09.007535 17254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:09.007618 17254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:09.007716 17254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:09.007822 17254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:09.014381 17254 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:37697
I20260808 07:13:09.014909 17254 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:09.018050 17495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:37697 every 8 connection(s)
I20260808 07:13:09.020524 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 17254
I20260808 07:13:09.020812 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:53949/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:39199
--local_ip_for_outbound_sockets=127.30.54.66
--tserver_master_addrs=127.30.54.126:33209
--webserver_port=40645
--webserver_interface=127.30.54.66
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.54.84:53949/kms with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:13:09 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:09.051658 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:09.018831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:33209 (local address 127.30.54.65:48985)
0808 07:13:09.019239 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:09.019281 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:09.019849 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:09.038071 (+ 18222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:09.038077 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:09.038351 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:09.038701 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:09.038709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.040552 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.040553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:09.041846 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:09.041850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.042707 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.042709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:09.042870 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:09.043502 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:09.043511 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:09.045203 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:09.049751 (+ 4548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:09.049758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:09.049760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:09.050063 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:09.050362 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:09.050363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:09.050364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:09.050504 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:09.050988 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:09.050995 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:09.051147 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 07:13:09.051249 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:13:09.051429 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":105,"threads_started":1}
I20260808 07:13:09.052775 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:09.018907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:48985 (local address 127.30.54.126:33209)
0808 07:13:09.025714 (+ 6807us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:09.025724 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:09.025767 (+ 43us) server_negotiation.cc:407] Connection header received
0808 07:13:09.025960 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:09.025969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:09.026035 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:09.038883 (+ 12848us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:09.038997 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.040412 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.041968 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.042575 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.043168 (+ 593us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:09.045330 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:09.045335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:09.045335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:09.045347 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:09.049642 (+ 4295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:09.050170 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:09.050173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:09.050173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:09.050280 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:09.050580 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:09.050581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:09.050582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:09.050892 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:09.052367 (+ 1475us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:09.052455 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:13:09.052578 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6721,"thread_start_us":95,"threads_started":1}
I20260808 07:13:09.053988 17497 heartbeater.cc:347] Connected to a master server at 127.30.54.126:33209
I20260808 07:13:09.054082 17497 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:09.054271 17497 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
W20260808 07:13:09.055213 17497 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:33209 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:13:09.055279 17497 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:09.055338 17497 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
I20260808 07:13:09.055747 17497 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:09.055827 17497 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
W20260808 07:13:09.056156 17497 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:13:09.141165 17501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:09.141422 17501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:09.141455 17501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:09.143041 17501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:09.143095 17501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:09.143131 17501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:13:09.145143 17501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.54.84:53949/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:39199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=40645
--tserver_master_addrs=127.30.54.126:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:09.145372 17501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:09.145612 17501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:09.152220 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:09.152485 17510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:09.152704 17517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:09.153656 17501 server_base.cc:1034] running on GCE node
I20260808 07:13:09.153940 17501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:09.154196 17501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:09.169620 17501 hybrid_clock.cc:648] HybridClock initialized: now 1786173189169577 us; error 60 us; skew 500 ppm
Aug 08 07:13:09 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:09.173133 17501 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:13:09.173655 17501 webserver.cc:460] Webserver started at http://127.30.54.66:40645/ using document root <none> and password file <none>
I20260808 07:13:09.173928 17501 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:09.174002 17501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:13:09 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:09.204339 17501 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.001s
I20260808 07:13:09.205451 17531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:09.205621 17501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:09.205720 17501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "dfc765f0ed594e6a8ad1e3941472b16e"
format_stamp: "Formatted at 2026-08-08 07:13:08 on dist-test-slave-znh6"
server_key: "e93cfaa825ec17d59770dea13e0cf097"
server_key_iv: "11ff40c461bdb0adad3092f6ef0ec07f"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:09.206252 17501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:09.243794 17501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:09.246497 17501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:09.246651 17501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:09.246948 17501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:09.247295 17501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:09.247342 17501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:09.247375 17501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:09.247406 17501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:09.255697 17501 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:39199
I20260808 07:13:09.255774 17717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:39199 every 8 connection(s)
I20260808 07:13:09.256210 17501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:13:09.260468 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 17501
I20260808 07:13:09.260887 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.54.84:53949/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:34743
--local_ip_for_outbound_sockets=127.30.54.67
--tserver_master_addrs=127.30.54.126:33209
--webserver_port=43813
--webserver_interface=127.30.54.67
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.54.84:53949/kms with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:13:09 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:09.266413 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:09.257082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:46745 (local address 127.30.54.126:33209)
0808 07:13:09.257376 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:09.257378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:09.257391 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:13:09.257881 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:09.257882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:09.257898 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:09.257973 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:09.258493 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.259340 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.260313 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.260531 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.260621 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:09.263211 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:09.263213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:09.263213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:09.263221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:09.264987 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:09.265413 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:09.265414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:09.265415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:09.265482 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:09.265746 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:09.265747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:09.265747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:09.265862 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:09.265934 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:09.266224 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 07:13:09.266321 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260808 07:13:09.266567 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:09.257088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:33209 (local address 127.30.54.66:46745)
0808 07:13:09.257277 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:09.257312 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:09.257761 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:09.258034 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:09.258038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:09.258201 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:09.258390 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:09.258394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.259436 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.259437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:09.260212 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:09.260214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.260705 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.260706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:09.260914 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:09.261791 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:09.261800 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:09.263111 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:09.265090 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:09.265096 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:09.265097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:09.265312 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:09.265556 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:09.265557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:09.265558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:09.265668 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:09.265938 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:09.265944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:09.266083 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 07:13:09.266187 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 07:13:09.266332 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260808 07:13:09.267127 17718 heartbeater.cc:347] Connected to a master server at 127.30.54.126:33209
I20260808 07:13:09.267232 17718 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:09.267444 17718 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
W20260808 07:13:09.268074 17718 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:33209 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:13:09.268158 17718 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:09.268234 17718 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
I20260808 07:13:09.268529 17718 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:09.268612 17718 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
W20260808 07:13:09.268939 17718 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:13:09.385540 17722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:09.385807 17722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:09.385848 17722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:09.387566 17722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:09.387622 17722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:09.387662 17722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:13:09.389808 17722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.54.84:53949/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:34743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=43813
--tserver_master_addrs=127.30.54.126:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:09.390054 17722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:09.390323 17722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:09.398268 17736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:09.398432 17722 server_base.cc:1034] running on GCE node
W20260808 07:13:09.398378 17729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:09.398942 17734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:09.399289 17722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:09.399537 17722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:09.400725 17722 hybrid_clock.cc:648] HybridClock initialized: now 1786173189400706 us; error 35 us; skew 500 ppm
Aug 08 07:13:09 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:09.403970 17722 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:13:09.404513 17722 webserver.cc:460] Webserver started at http://127.30.54.67:43813/ using document root <none> and password file <none>
I20260808 07:13:09.404809 17722 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:09.404903 17722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:13:09 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:09.420106 17722 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:09.421065 17749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:09.421221 17722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:09.421304 17722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "b2d6ac491e1b4f85b88969a62b30560e"
format_stamp: "Formatted at 2026-08-08 07:13:08 on dist-test-slave-znh6"
server_key: "11f605cd2a2ecb34532901b5c78b3e38"
server_key_iv: "d3151a279bd2075266d1853f00f8bb01"
server_key_version: "kuduclusterkey@0"
I20260808 07:13:09.421797 17722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:09.432677 17722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:09.434787 17722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:09.434974 17722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:09.435225 17722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:09.435647 17722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:09.435704 17722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:09.435746 17722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:09.435773 17722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:09.443482 17722 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:34743
I20260808 07:13:09.443928 17722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:13:09.444571 17923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:34743 every 8 connection(s)
I20260808 07:13:09.449891 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 17722
Aug 08 07:13:09 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:09.455974 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:09.446260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:56525 (local address 127.30.54.126:33209)
0808 07:13:09.446310 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:09.446312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:09.446322 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:09.446349 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:09.446350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:09.446360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:09.446434 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:09.447165 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.447939 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.449325 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.449623 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.449725 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:09.452367 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:09.452369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:09.452369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:09.452378 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:09.454061 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:09.454559 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:09.454560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:09.454560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:09.454628 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:09.454938 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:09.454939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:09.454939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:09.455092 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:09.455127 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:09.455765 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 07:13:09.455847 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:13:09.456543 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:09.444675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:33209 (local address 127.30.54.67:56525)
0808 07:13:09.444835 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:09.444878 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:09.445494 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:09.446529 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:09.446534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:09.446740 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:09.447046 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:09.447053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.448079 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.448080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:09.449221 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:09.449224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.449812 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.449814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:09.449969 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:09.450512 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:09.450522 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:09.452248 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:09.454156 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:09.454164 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:09.454166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:09.454460 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:09.454699 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:09.454700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:09.454700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:09.454856 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:09.455414 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:09.455421 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:09.455586 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 07:13:09.456112 (+ 526us) client_negotiation.cc:240] Negotiation successful
0808 07:13:09.456295 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260808 07:13:09.456976 17925 heartbeater.cc:347] Connected to a master server at 127.30.54.126:33209
I20260808 07:13:09.457068 17925 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:09.457265 17925 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
W20260808 07:13:09.457876 17925 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:33209 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:13:09.457953 17925 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:09.458014 17925 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
I20260808 07:13:09.458313 17925 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:09.458380 17925 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
W20260808 07:13:09.458662 17925 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:13:09.460076 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:09.450283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43898 (local address 127.30.54.126:33209)
0808 07:13:09.451055 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:09.451057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:09.451248 (+ 191us) server_negotiation.cc:407] Connection header received
0808 07:13:09.451300 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:09.451301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:09.451313 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:09.451390 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:09.452126 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.452889 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.454579 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:09.454916 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:09.454973 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:09.456588 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:09.456589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:09.456590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:09.456597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:09.458373 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:09.458795 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:09.458796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:09.458797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:09.458883 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:09.459533 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:09.459540 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:09.459541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:09.459641 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:09.459827 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:09.459864 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:13:09.459959 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":65,"threads_started":1}
Aug 08 07:13:09 dist-test-slave-znh6 krb5kdc[15663](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786173189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:10.056358 17497 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:10.056514 17497 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
I20260808 07:13:10.056900 17190 ts_manager.cc:194] Registered new tserver with Master: b40059ead5b649b285b663539c009c5a (127.30.54.65:37697)
I20260808 07:13:10.057962 17190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:48985
I20260808 07:13:10.269088 17718 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:10.269246 17718 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
I20260808 07:13:10.269539 17190 ts_manager.cc:194] Registered new tserver with Master: dfc765f0ed594e6a8ad1e3941472b16e (127.30.54.66:39199)
I20260808 07:13:10.270030 17190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:46745
Aug 08 07:13:10 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
I20260808 07:13:10.458845 17925 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:10.459000 17925 heartbeater.cc:511] Master 127.30.54.126:33209 requested a full tablet report, sending...
I20260808 07:13:10.459290 17190 ts_manager.cc:194] Registered new tserver with Master: b2d6ac491e1b4f85b88969a62b30560e (127.30.54.67:34743)
I20260808 07:13:10.459913 17190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:56525
I20260808 07:13:10.469597 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:10.482187 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.470637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.30.54.126:33209)
0808 07:13:10.470919 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:10.470921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:10.470933 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:13:10.470957 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:10.470958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:10.470970 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:10.471050 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:10.471751 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.472529 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.473440 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.473765 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.473837 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.478929 (+ 5092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:10.478930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:10.478931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:10.478939 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:10.480821 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:10.481231 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:10.481232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:10.481233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:10.481299 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:10.481518 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:10.481518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:10.481519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:10.481653 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:10.481751 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:10.481893 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 07:13:10.481984 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":93,"threads_started":1}
I20260808 07:13:10.484750 17190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:13:10.486943 17190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:13:10.498545 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.492621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.30.54.66:39199)
0808 07:13:10.492797 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:10.492809 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:10.493562 (+ 753us) server_negotiation.cc:407] Connection header received
0808 07:13:10.493968 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:10.493971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:10.493988 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:10.494040 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:10.494358 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.495523 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.497000 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.497963 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.498054 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.498110 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:10.498388 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 07:13:10.498432 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260808 07:13:10.498807 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.492398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:39199 (local address 127.0.0.1:60216)
0808 07:13:10.493488 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:10.493499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:10.493862 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:10.494208 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:10.494211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:10.494213 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:10.494279 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.494280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.495622 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.495623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.496898 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.496901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.498177 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.498178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.498261 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.498274 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:13:10.498662 (+ 388us) client_negotiation.cc:240] Negotiation successful
0808 07:13:10.498726 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"thread_start_us":448,"threads_started":1}
I20260808 07:13:10.499593 17604 tablet_service.cc:1467] Processing CreateTablet for tablet 8a41bf7783c843d2b86c0c0448727b45 (DEFAULT_TABLE table=test-table [id=3dd6be807d4b43f1b23e576243682dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:10.500051 17604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a41bf7783c843d2b86c0c0448727b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:10.503203 17984 tablet_bootstrap.cc:492] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e: Bootstrap starting.
I20260808 07:13:10.504410 17984 tablet_bootstrap.cc:654] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:10.505070 17984 log.cc:824] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:10.507171 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.492950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:34743 (local address 127.0.0.1:60002)
0808 07:13:10.498959 (+ 6009us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:10.498969 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:10.499039 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:10.503278 (+ 4239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:10.503279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:10.503281 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:10.503321 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.503322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.505006 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.505007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.505998 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.506001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.506935 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.506936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.506993 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.506997 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:10.507044 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:13:10.507090 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5991,"thread_start_us":114,"threads_started":1}
I20260808 07:13:10.507387 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.493227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.30.54.67:34743)
0808 07:13:10.493720 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:10.493725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:10.499102 (+ 5377us) server_negotiation.cc:407] Connection header received
0808 07:13:10.503073 (+ 3971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:10.503076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:10.503093 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:10.503168 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:10.503407 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.504859 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.506091 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.506771 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.506849 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.506903 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:10.507262 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 07:13:10.507311 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":57,"threads_started":1}
I20260808 07:13:10.508491 17826 tablet_service.cc:1467] Processing CreateTablet for tablet 8a41bf7783c843d2b86c0c0448727b45 (DEFAULT_TABLE table=test-table [id=3dd6be807d4b43f1b23e576243682dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:10.509163 17826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a41bf7783c843d2b86c0c0448727b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:10.515829 17987 tablet_bootstrap.cc:492] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: Bootstrap starting.
I20260808 07:13:10.517068 17987 tablet_bootstrap.cc:654] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:10.517956 17987 log.cc:824] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:10.519503 17987 tablet_bootstrap.cc:492] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: No bootstrap required, opened a new log
I20260808 07:13:10.519596 17987 ts_tablet_manager.cc:1397] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:13:10.519616 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.492332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:37697 (local address 127.0.0.1:40048)
0808 07:13:10.492944 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:10.492995 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:10.493494 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:10.499577 (+ 6083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:10.499578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:10.499581 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:10.499724 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.499726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.515995 (+ 16269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.515996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.517620 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.517623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.519317 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.519318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.519443 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.519448 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:13:10.519481 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 07:13:10.519520 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":167,"threads_started":1}
I20260808 07:13:10.520097 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.492452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40048 (local address 127.30.54.65:37697)
0808 07:13:10.499399 (+ 6947us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:10.499403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:10.499415 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:13:10.499450 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:10.499452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:10.499467 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:10.499553 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:10.514684 (+ 15131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.515802 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.517775 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.519212 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.519783 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.519852 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:10.519878 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 07:13:10.519915 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6909,"thread_start_us":52,"threads_started":1}
I20260808 07:13:10.520848 17396 tablet_service.cc:1467] Processing CreateTablet for tablet 8a41bf7783c843d2b86c0c0448727b45 (DEFAULT_TABLE table=test-table [id=3dd6be807d4b43f1b23e576243682dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:10.521131 17984 tablet_bootstrap.cc:492] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e: No bootstrap required, opened a new log
I20260808 07:13:10.521202 17984 ts_tablet_manager.cc:1397] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.002s
I20260808 07:13:10.521224 17396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a41bf7783c843d2b86c0c0448727b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:10.525763 17987 raft_consensus.cc:348] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } }
I20260808 07:13:10.525878 17984 raft_consensus.cc:348] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } }
I20260808 07:13:10.526063 17984 raft_consensus.cc:374] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:10.526108 17984 raft_consensus.cc:729] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc765f0ed594e6a8ad1e3941472b16e, State: Initialized, Role: FOLLOWER
I20260808 07:13:10.526101 17987 raft_consensus.cc:374] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:10.526242 17984 consensus_queue.cc:260] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } }
I20260808 07:13:10.526283 17987 raft_consensus.cc:729] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2d6ac491e1b4f85b88969a62b30560e, State: Initialized, Role: FOLLOWER
I20260808 07:13:10.526556 17984 ts_tablet_manager.cc:1428] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260808 07:13:10.526543 17987 consensus_queue.cc:260] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } }
I20260808 07:13:10.526676 17718 heartbeater.cc:503] Master 127.30.54.126:33209 was elected leader, sending a full tablet report...
I20260808 07:13:10.526932 17987 ts_tablet_manager.cc:1428] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 07:13:10.527091 17925 heartbeater.cc:503] Master 127.30.54.126:33209 was elected leader, sending a full tablet report...
I20260808 07:13:10.528163 17991 tablet_bootstrap.cc:492] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a: Bootstrap starting.
I20260808 07:13:10.529207 17991 tablet_bootstrap.cc:654] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:10.529829 17991 log.cc:824] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:10.530889 17991 tablet_bootstrap.cc:492] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a: No bootstrap required, opened a new log
I20260808 07:13:10.530965 17991 ts_tablet_manager.cc:1397] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:13:10.534624 17991 raft_consensus.cc:348] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } }
I20260808 07:13:10.534781 17991 raft_consensus.cc:374] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:10.534837 17991 raft_consensus.cc:729] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b40059ead5b649b285b663539c009c5a, State: Initialized, Role: FOLLOWER
I20260808 07:13:10.534966 17991 consensus_queue.cc:260] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } }
I20260808 07:13:10.535266 17991 ts_tablet_manager.cc:1428] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 07:13:10.535420 17497 heartbeater.cc:503] Master 127.30.54.126:33209 was elected leader, sending a full tablet report...
Aug 08 07:13:10 dist-test-slave-znh6 krb5kdc[15663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for HTTP/127.30.54.84@KRBTEST.COM
W20260808 07:13:10.695353 17926 tablet.cc:2369] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:13:10.757665 17719 tablet.cc:2369] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:13:10.770358 17498 tablet.cc:2369] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:13:10.780088 17992 raft_consensus.cc:482] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:10.780280 17992 raft_consensus.cc:504] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } }
I20260808 07:13:10.780779 17992 leader_election.cc:288] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2d6ac491e1b4f85b88969a62b30560e (127.30.54.67:34743), b40059ead5b649b285b663539c009c5a (127.30.54.65:37697)
I20260808 07:13:10.795384 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.781184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:56807 (local address 127.30.54.67:34743)
0808 07:13:10.781246 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:10.781249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:10.781679 (+ 430us) server_negotiation.cc:407] Connection header received
0808 07:13:10.782317 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:10.782318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:10.782328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:10.782359 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:10.782689 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.783494 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.786173 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.786784 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.786871 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.786882 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:10.795258 (+ 8376us) server_negotiation.cc:299] Negotiation successful
0808 07:13:10.795311 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:13:10.795693 18001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.780948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:34743 (local address 127.30.54.66:56807)
0808 07:13:10.781596 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:10.781609 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:10.782202 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:10.782437 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:10.782438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:10.782441 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:10.782592 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.782595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.784653 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.784654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.786059 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.786063 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.795009 (+ 8946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.795010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.795130 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.795135 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:13:10.795526 (+ 391us) client_negotiation.cc:240] Negotiation successful
0808 07:13:10.795571 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":416,"threads_started":1}
I20260808 07:13:10.796124 17859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a41bf7783c843d2b86c0c0448727b45" candidate_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2d6ac491e1b4f85b88969a62b30560e" is_pre_election: true
I20260808 07:13:10.796320 17859 raft_consensus.cc:2454] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc765f0ed594e6a8ad1e3941472b16e in term 0.
I20260808 07:13:10.796614 17534 leader_election.cc:302] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2d6ac491e1b4f85b88969a62b30560e, dfc765f0ed594e6a8ad1e3941472b16e; no voters:
I20260808 07:13:10.796746 17992 raft_consensus.cc:2781] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:10.796783 17992 raft_consensus.cc:482] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:10.796803 17992 raft_consensus.cc:3037] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:10.798120 17992 raft_consensus.cc:504] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } }
I20260808 07:13:10.798256 17992 leader_election.cc:288] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [CANDIDATE]: Term 1 election: Requested vote from peers b2d6ac491e1b4f85b88969a62b30560e (127.30.54.67:34743), b40059ead5b649b285b663539c009c5a (127.30.54.65:37697)
I20260808 07:13:10.798604 17859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a41bf7783c843d2b86c0c0448727b45" candidate_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2d6ac491e1b4f85b88969a62b30560e"
I20260808 07:13:10.798686 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.781240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:47651 (local address 127.30.54.65:37697)
0808 07:13:10.781287 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:10.781289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:10.793716 (+ 12427us) server_negotiation.cc:407] Connection header received
0808 07:13:10.793829 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:10.793830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:10.793841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:10.793873 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:10.794215 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.794947 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.797568 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.798224 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.798277 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.798287 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:10.798572 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 07:13:10.798620 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:13:10.798678 17859 raft_consensus.cc:3037] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:10.798982 18003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.780985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:37697 (local address 127.30.54.66:47651)
0808 07:13:10.793633 (+ 12648us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:10.793646 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:10.793748 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:10.793943 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:10.793944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:10.793946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:10.794116 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.794118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.795037 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.795038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.797392 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:10.797395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.798350 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.798350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:10.798480 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.798484 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:10.798797 (+ 313us) client_negotiation.cc:240] Negotiation successful
0808 07:13:10.798866 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12612,"thread_start_us":511,"threads_started":1}
I20260808 07:13:10.799410 17416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a41bf7783c843d2b86c0c0448727b45" candidate_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40059ead5b649b285b663539c009c5a" is_pre_election: true
I20260808 07:13:10.799598 17416 raft_consensus.cc:2454] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc765f0ed594e6a8ad1e3941472b16e in term 0.
I20260808 07:13:10.799772 17415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a41bf7783c843d2b86c0c0448727b45" candidate_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b40059ead5b649b285b663539c009c5a"
I20260808 07:13:10.799823 17415 raft_consensus.cc:3037] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:10.799960 17859 raft_consensus.cc:2454] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc765f0ed594e6a8ad1e3941472b16e in term 1.
I20260808 07:13:10.800392 17534 leader_election.cc:302] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2d6ac491e1b4f85b88969a62b30560e, dfc765f0ed594e6a8ad1e3941472b16e; no voters:
I20260808 07:13:10.800539 17992 raft_consensus.cc:2781] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:10.800709 17992 raft_consensus.cc:686] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 1 LEADER]: Becoming Leader. State: Replica: dfc765f0ed594e6a8ad1e3941472b16e, State: Running, Role: LEADER
I20260808 07:13:10.800722 17415 raft_consensus.cc:2454] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc765f0ed594e6a8ad1e3941472b16e in term 1.
I20260808 07:13:10.800794 17992 consensus_queue.cc:237] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } }
I20260808 07:13:10.801656 17190 catalog_manager.cc:5495] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e reported cstate change: term changed from 0 to 1, leader changed from <none> to dfc765f0ed594e6a8ad1e3941472b16e (127.30.54.66). New cstate: current_term: 1 leader_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc765f0ed594e6a8ad1e3941472b16e" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 39199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 } health_report { overall_health: UNKNOWN } } }
I20260808 07:13:10.859498 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.855925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.30.54.66:39199)
0808 07:13:10.855988 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:10.855991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:10.856005 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:13:10.856049 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:10.856050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:10.856072 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:10.856174 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:10.856353 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.857119 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.858207 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:10.858513 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:10.858584 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:10.858802 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:10.859001 (+ 199us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:10.859094 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:10.859400 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 07:13:10.859421 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:13:10.864445 17415 raft_consensus.cc:1261] T 8a41bf7783c843d2b86c0c0448727b45 P b40059ead5b649b285b663539c009c5a [term 1 FOLLOWER]: Refusing update from remote peer dfc765f0ed594e6a8ad1e3941472b16e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:10.864986 18004 consensus_queue.cc:1035] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b40059ead5b649b285b663539c009c5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 37697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:10.867066 17859 raft_consensus.cc:1261] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [term 1 FOLLOWER]: Refusing update from remote peer dfc765f0ed594e6a8ad1e3941472b16e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:10.867405 18004 consensus_queue.cc:1035] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2d6ac491e1b4f85b88969a62b30560e" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:10.871860 18007 mvcc.cc:205] Tried to move back new op lower bound from 7316165389776093184 to 7316165389520728064. Current Snapshot: MvccSnapshot[applied={T|T < 7316165389776093184}]
I20260808 07:13:10.872421 18011 mvcc.cc:205] Tried to move back new op lower bound from 7316165389776093184 to 7316165389520728064. Current Snapshot: MvccSnapshot[applied={T|T < 7316165389776093184}]
I20260808 07:13:10.874897 18008 mvcc.cc:205] Tried to move back new op lower bound from 7316165389776093184 to 7316165389520728064. Current Snapshot: MvccSnapshot[applied={T|T < 7316165389776093184}]
I20260808 07:13:10.877552 17190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:10.877650 17190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:10.878551 17190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea4bf5cf3eaf4728a63e853f3c0757fc: Sending DeleteTablet for 3 replicas of tablet 8a41bf7783c843d2b86c0c0448727b45
I20260808 07:13:10.879043 17604 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a41bf7783c843d2b86c0c0448727b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:10 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:60216
I20260808 07:13:10.879132 17826 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a41bf7783c843d2b86c0c0448727b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:10 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:60002
I20260808 07:13:10.879271 17396 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a41bf7783c843d2b86c0c0448727b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:10 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:40048
I20260808 07:13:10.879633 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 17254
I20260808 07:13:10.890969 18023 tablet_replica.cc:330] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: stopping tablet replica
I20260808 07:13:10.891091 18023 raft_consensus.cc:2229] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:10.891222 18023 raft_consensus.cc:2258] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:10.891790 18023 ts_tablet_manager.cc:1905] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:10.893934 18023 ts_tablet_manager.cc:1918] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:10.894013 18023 log.cc:1196] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/8a41bf7783c843d2b86c0c0448727b45
I20260808 07:13:10.894299 18023 ts_tablet_manager.cc:1939] T 8a41bf7783c843d2b86c0c0448727b45 P b2d6ac491e1b4f85b88969a62b30560e: Deleting consensus metadata
I20260808 07:13:10.898411 17172 catalog_manager.cc:4841] TS b2d6ac491e1b4f85b88969a62b30560e (127.30.54.67:34743): tablet 8a41bf7783c843d2b86c0c0448727b45 (table test-table [id=3dd6be807d4b43f1b23e576243682dd5]) successfully deleted
I20260808 07:13:10.898936 18022 tablet_replica.cc:330] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e: stopping tablet replica
I20260808 07:13:10.899065 18022 raft_consensus.cc:2229] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:13:10.899186 18022 raft_consensus.cc:2258] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 07:13:10.905287 17171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:37697 (error 108)
I20260808 07:13:10.907253 18022 ts_tablet_manager.cc:1905] T 8a41bf7783c843d2b86c0c0448727b45 P dfc765f0ed594e6a8ad1e3941472b16e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:10.907473 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 17501
I20260808 07:13:10.919312 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:10.919140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:37697 (local address 127.0.0.1:40058)
0808 07:13:10.919216 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:10.919272 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:37697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 07:13:10.919476 17171 catalog_manager.cc:4584] TS b40059ead5b649b285b663539c009c5a (127.30.54.65:37697): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a41bf7783c843d2b86c0c0448727b45: Network error: Client connection negotiation failed: client connection to 127.30.54.65:37697: connect: Connection refused (error 111)
W20260808 07:13:10.923952 17981 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:13:10.923695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:39199 (local address 127.0.0.1:60234)
0808 07:13:10.923754 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:10.923765 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:10.923801 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:10.923905 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.66:39199: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:13:10.924403 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 17722
I20260808 07:13:10.931500 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 17141
I20260808 07:13:10.940363 30937 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:13:11.294685 30937 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:13:11.294800 30937 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:13:11.638877 30937 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:13:11.639 UTC [15708] LOG: received smart shutdown request
2026-08-08 07:13:11.643 UTC [15708] LOG: background worker "logical replication launcher" (PID 15728) exited with exit code 1
2026-08-08 07:13:11.644 UTC [15713] LOG: shutting down
2026-08-08 07:13:11.810 UTC [15708] LOG: database system is shut down
2026-08-08T07:13:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49079 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:13:11 dist-test-slave-znh6 krb5kdc[18049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:13:11 dist-test-slave-znh6 krb5kdc[18049](info): set up 2 sockets
Aug 08 07:13:11 dist-test-slave-znh6 krb5kdc[18049](info): commencing operation
krb5kdc: starting...
W20260808 07:13:13.141453 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:13:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:13:13.260069 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45657
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:45657,127.30.54.125:40195,127.30.54.124:46807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:13.340370 18074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:13.340610 18074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:13.340646 18074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:13.341737 18074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:13.341804 18074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:13.341825 18074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:13.341845 18074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:13.341877 18074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:13.343710 18074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:45657,127.30.54.125:40195,127.30.54.124:46807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:13.344008 18074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:13.344235 18074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:13.349187 18085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:13.349201 18081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:13.349185 18082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:13.349443 18074 server_base.cc:1034] running on GCE node
I20260808 07:13:13.349666 18074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:13.349942 18074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:13.351107 18074 hybrid_clock.cc:648] HybridClock initialized: now 1786173193351087 us; error 38 us; skew 500 ppm
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:13.353549 18074 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:13:13.353933 18074 webserver.cc:460] Webserver started at http://127.30.54.126:33093/ using document root <none> and password file <none>
I20260808 07:13:13.354117 18074 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:13.354156 18074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:13.354239 18074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:13.355290 18074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f"
format_stamp: "Formatted at 2026-08-08 07:13:13 on dist-test-slave-znh6"
server_key: "18613f8a0c09ebd6d32b611b1cf09fbc"
server_key_iv: "6b28ab55a668c1ca3d9ddbc8a4b3dcd4"
server_key_version: "encryptionkey@0"
I20260808 07:13:13.355603 18074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f"
format_stamp: "Formatted at 2026-08-08 07:13:13 on dist-test-slave-znh6"
server_key: "18613f8a0c09ebd6d32b611b1cf09fbc"
server_key_iv: "6b28ab55a668c1ca3d9ddbc8a4b3dcd4"
server_key_version: "encryptionkey@0"
I20260808 07:13:13.356858 18074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:13:13.357525 18093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:13.357697 18074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:13.357802 18074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f"
format_stamp: "Formatted at 2026-08-08 07:13:13 on dist-test-slave-znh6"
server_key: "18613f8a0c09ebd6d32b611b1cf09fbc"
server_key_iv: "6b28ab55a668c1ca3d9ddbc8a4b3dcd4"
server_key_version: "encryptionkey@0"
I20260808 07:13:13.357873 18074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:13.378010 18074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:13.379956 18074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:13.380121 18074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:13.384168 18074 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:45657
I20260808 07:13:13.384208 18179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:45657 every 8 connection(s)
I20260808 07:13:13.384562 18074 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:13.385115 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 18074
I20260808 07:13:13.385223 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:13:13.385430 30937 external_mini_cluster.cc:1428] Setting key 324b15a02623c1fcf9014b3136dab596
I20260808 07:13:13.385667 18180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:13.387306 18180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.54.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.125@KRBTEST.COM" created.
I20260808 07:13:13.397930 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.388024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39736 (local address 127.30.54.126:45657)
0808 07:13:13.388181 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.388192 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.388330 (+ 138us) server_negotiation.cc:407] Connection header received
0808 07:13:13.388822 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.388825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.388848 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.388955 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.389504 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.390946 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.391820 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.392361 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.392483 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.394778 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.394787 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.394787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.394796 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.396335 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.396707 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.396710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.396710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.396791 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.397056 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.397057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.397058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.397276 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.397350 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.397535 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.397737 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260808 07:13:13.397938 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.387964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45657 (local address 127.0.0.1:39736)
0808 07:13:13.388182 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.388220 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:13.388665 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:13.388975 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:13.388977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:13.389224 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:13.389404 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.389407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.391041 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.391042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.391725 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.391727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.392449 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.392450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.392507 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.393057 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:13.393065 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:13.394662 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:13.396422 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.396428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.396430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.396621 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.396865 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.396865 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.396866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.396978 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.397348 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:13.397353 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:13.397408 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 07:13:13.397477 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:13:13.397657 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":97,"threads_started":1}
I20260808 07:13:13.398731 18180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:13.399117 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.398985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:40195 (local address 127.0.0.1:46326)
0808 07:13:13.399036 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.399085 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:13:13.399263 18100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
I20260808 07:13:13.399822 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.399702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:40195 (local address 127.0.0.1:46336)
0808 07:13:13.399755 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.399792 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:13:13.399960 18180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.125:40195: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab.
Entry for principal kudu/127.30.54.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab.
I20260808 07:13:13.413898 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.125:40195
--webserver_interface=127.30.54.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab
--principal=kudu/127.30.54.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:45657,127.30.54.125:40195,127.30.54.124:46807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:13.449225 18180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } attempt: 1
I20260808 07:13:13.449743 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.449603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:40195 (local address 127.0.0.1:46342)
0808 07:13:13.449672 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.449708 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 07:13:13.450219 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.450120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:40195 (local address 127.0.0.1:46358)
0808 07:13:13.450169 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.450198 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 07:13:13.450340 18180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.125:40195: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
W20260808 07:13:13.496016 18188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:13.496238 18188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:13.496275 18188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:13.497372 18188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:13.497434 18188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:13.497460 18188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:13.497489 18188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:13.497514 18188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:13.499091 18188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:45657,127.30.54.125:40195,127.30.54.124:46807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab
--principal=kudu/127.30.54.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.125:40195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.54.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:13.499348 18188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:13.499560 18188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:13.504231 18201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:13.504231 18194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:13.504267 18197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:13.504303 18188 server_base.cc:1034] running on GCE node
I20260808 07:13:13.504673 18188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:13.504981 18188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:13.506134 18188 hybrid_clock.cc:648] HybridClock initialized: now 1786173193506118 us; error 46 us; skew 500 ppm
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:13.508750 18188 init.cc:377] Logged in from keytab as kudu/127.30.54.125@KRBTEST.COM (short username kudu)
I20260808 07:13:13.509120 18188 webserver.cc:460] Webserver started at http://127.30.54.125:40211/ using document root <none> and password file <none>
I20260808 07:13:13.509287 18188 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:13.509326 18188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:13.509413 18188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:13.510357 18188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/data/instance:
uuid: "442fd9e386694bd3bb0d489d5fa52d11"
format_stamp: "Formatted at 2026-08-08 07:13:13 on dist-test-slave-znh6"
server_key: "a93d42d5a2cad83b4a0c9f35c1a5e244"
server_key_iv: "7df0a3963453f68bf5f329a9c3e800d0"
server_key_version: "encryptionkey@0"
I20260808 07:13:13.510660 18188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/wal/instance:
uuid: "442fd9e386694bd3bb0d489d5fa52d11"
format_stamp: "Formatted at 2026-08-08 07:13:13 on dist-test-slave-znh6"
server_key: "a93d42d5a2cad83b4a0c9f35c1a5e244"
server_key_iv: "7df0a3963453f68bf5f329a9c3e800d0"
server_key_version: "encryptionkey@0"
I20260808 07:13:13.511971 18188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:13:13.512634 18211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:13.512830 18188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:13.512918 18188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/wal
uuid: "442fd9e386694bd3bb0d489d5fa52d11"
format_stamp: "Formatted at 2026-08-08 07:13:13 on dist-test-slave-znh6"
server_key: "a93d42d5a2cad83b4a0c9f35c1a5e244"
server_key_iv: "7df0a3963453f68bf5f329a9c3e800d0"
server_key_version: "encryptionkey@0"
I20260808 07:13:13.512982 18188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:13.518492 18180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } attempt: 2
I20260808 07:13:13.518947 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.518777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:40195 (local address 127.0.0.1:46368)
0808 07:13:13.518869 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.518910 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 07:13:13.519280 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.519193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:40195 (local address 127.0.0.1:46384)
0808 07:13:13.519233 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.519257 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 07:13:13.519368 18180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.125:40195: Network error: Client connection negotiation failed: client connection to 127.30.54.125:40195: connect: Connection refused (error 111)
I20260808 07:13:13.530292 18188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:13.532482 18188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:13.532637 18188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:13.536444 18188 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.125:40195
I20260808 07:13:13.536494 18289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.125:40195 every 8 connection(s)
I20260808 07:13:13.536825 18188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
I20260808 07:13:13.537673 18290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:13.539150 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 18188
I20260808 07:13:13.539255 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-1/wal/instance
I20260808 07:13:13.539455 30937 external_mini_cluster.cc:1428] Setting key 831768ff88e0f2116026b51feb8fc86e
I20260808 07:13:13.539290 18290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.54.124@KRBTEST.COM; defaulting to no policy
I20260808 07:13:13.548262 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.540067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.30.54.126:45657)
0808 07:13:13.540131 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.540133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.540309 (+ 176us) server_negotiation.cc:407] Connection header received
0808 07:13:13.540730 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.540731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.540746 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.540816 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.541400 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.541903 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.542916 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.543126 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.543202 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.545197 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.545199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.545199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.545207 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.546857 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.547223 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.547225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.547225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.547290 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.547568 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.547569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.547569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.547710 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.547780 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.548041 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.548131 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:13:13.548280 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.539995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45657 (local address 127.0.0.1:39748)
0808 07:13:13.540182 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.540221 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:13.540648 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:13.540886 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:13.540889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:13.541114 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:13.541303 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.541307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.541997 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.541999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.542785 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.542787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.543209 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.543211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.543341 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.543785 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:13.543793 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:13.545107 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:13.546931 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.546938 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.546940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.547150 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.547391 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.547392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.547393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.547500 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.547784 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:13.547789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:13.547893 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 07:13:13.547999 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:13:13.548127 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260808 07:13:13.548789 18290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.124@KRBTEST.COM" created.
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
I20260808 07:13:13.555878 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.549036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:40195 (local address 127.0.0.1:46388)
0808 07:13:13.549091 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.549103 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:13.549165 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:13.549345 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:13.549346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:13.549458 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:13.549509 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.549510 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.550253 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.550254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.550879 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.550882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.551352 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.551353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.551418 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.551817 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:13.551821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:13.552906 (+ 1085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:13.554943 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.554944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.554944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.555103 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.555313 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.555314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.555314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.555361 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.555671 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:13.555672 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:13.555705 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 07:13:13.555754 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:13:13.555819 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:13:13.556035 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.549070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.30.54.125:40195)
0808 07:13:13.549215 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.549220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.549237 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:13:13.549262 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.549264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.549274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.549333 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.549569 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.550191 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.550966 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.551260 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.551352 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.552987 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.552996 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.552996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.553002 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.554868 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.555184 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.555187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.555188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.555249 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.555415 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.555416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.555416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.555605 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.555697 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.555801 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.555889 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260808 07:13:13.556576 18290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:13.556876 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.556776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39226)
0808 07:13:13.556817 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.556860 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 07:13:13.557021 18217 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
I20260808 07:13:13.557428 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.557348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39228)
0808 07:13:13.557390 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.557413 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:13:13.557543 18290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46807: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab.
Entry for principal kudu/127.30.54.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab.
I20260808 07:13:13.566066 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.124:46807
--webserver_interface=127.30.54.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab
--principal=kudu/127.30.54.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:45657,127.30.54.125:40195,127.30.54.124:46807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:13.606760 18290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } attempt: 1
I20260808 07:13:13.607273 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.607145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39238)
0808 07:13:13.607205 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.607243 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:13:13.607604 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.607508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39250)
0808 07:13:13.607550 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.607575 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 07:13:13.607689 18290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46807: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
I20260808 07:13:13.610498 18180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } attempt: 3
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
I20260808 07:13:13.617421 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.610864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46390 (local address 127.30.54.125:40195)
0808 07:13:13.610923 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.610925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.610952 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:13:13.610985 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.610986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.610996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.611060 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.611382 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.611872 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.612573 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.612815 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.612890 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.615043 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.615044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.615044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.615050 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.616384 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.616768 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.616769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.616770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.616820 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.617010 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.617011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.617011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.617109 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.617158 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.617312 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.617369 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:13:13.617494 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.610775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:40195 (local address 127.0.0.1:46390)
0808 07:13:13.610845 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.610856 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:13.610918 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:13.611067 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:13.611068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:13.611232 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:13.611296 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.611298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.611955 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.611956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.612491 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.612493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.612887 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.612889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.612972 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.613448 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:13.613452 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:13.614965 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:13.616475 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.616476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.616477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.616696 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.616880 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.616881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.616881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.616950 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.617166 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:13.617167 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:13.617221 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 07:13:13.617298 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:13:13.617398 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 07:13:13.617904 18180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:13.618203 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.618118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39254)
0808 07:13:13.618152 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.618181 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 07:13:13.618484 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.618429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39262)
0808 07:13:13.618451 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.618472 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260808 07:13:13.618578 18180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46807: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
W20260808 07:13:13.649055 18299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:13.649248 18299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:13.649271 18299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 07:13:13.649708 18180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } attempt: 1
I20260808 07:13:13.650171 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.650036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39272)
0808 07:13:13.650096 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.650141 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:13:13.650584 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.650495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39288)
0808 07:13:13.650539 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.650567 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:13:13.650691 18299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:13.650696 18180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46807: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
W20260808 07:13:13.650746 18299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:13.650769 18299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:13.650789 18299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:13.650830 18299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:13.652338 18299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:45657,127.30.54.125:40195,127.30.54.124:46807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab
--principal=kudu/127.30.54.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.124:46807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.54.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:13.652571 18299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:13.652809 18299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:13.657681 18307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:13.657802 18299 server_base.cc:1034] running on GCE node
W20260808 07:13:13.657699 18311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:13.657924 18305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:13.658182 18299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:13.658471 18299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:13.659631 18299 hybrid_clock.cc:648] HybridClock initialized: now 1786173193659615 us; error 26 us; skew 500 ppm
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:13.662248 18299 init.cc:377] Logged in from keytab as kudu/127.30.54.124@KRBTEST.COM (short username kudu)
I20260808 07:13:13.662716 18299 webserver.cc:460] Webserver started at http://127.30.54.124:43161/ using document root <none> and password file <none>
I20260808 07:13:13.662985 18299 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:13.663034 18299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:13.663194 18299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:13.664242 18299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/data/instance:
uuid: "29183a7931744372b9f1c19c3afd34ed"
format_stamp: "Formatted at 2026-08-08 07:13:13 on dist-test-slave-znh6"
server_key: "f872c2c32d8e82f1d903657d1db696eb"
server_key_iv: "a6bf1d0bbbca9e7e4714ec7b905a9607"
server_key_version: "encryptionkey@0"
I20260808 07:13:13.664594 18299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/wal/instance:
uuid: "29183a7931744372b9f1c19c3afd34ed"
format_stamp: "Formatted at 2026-08-08 07:13:13 on dist-test-slave-znh6"
server_key: "f872c2c32d8e82f1d903657d1db696eb"
server_key_iv: "a6bf1d0bbbca9e7e4714ec7b905a9607"
server_key_version: "encryptionkey@0"
I20260808 07:13:13.665874 18299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:13:13.666584 18324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:13.666743 18299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:13.666872 18299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/wal
uuid: "29183a7931744372b9f1c19c3afd34ed"
format_stamp: "Formatted at 2026-08-08 07:13:13 on dist-test-slave-znh6"
server_key: "f872c2c32d8e82f1d903657d1db696eb"
server_key_iv: "a6bf1d0bbbca9e7e4714ec7b905a9607"
server_key_version: "encryptionkey@0"
I20260808 07:13:13.666939 18299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:13.675824 18290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } attempt: 2
I20260808 07:13:13.676204 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.676086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39298)
0808 07:13:13.676142 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.676177 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:13:13.676615 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.676514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39308)
0808 07:13:13.676555 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.676589 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 07:13:13.676738 18290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46807: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46807: connect: Connection refused (error 111)
I20260808 07:13:13.684548 18299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:13.686278 18299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:13.686426 18299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:13.690258 18299 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.124:46807
I20260808 07:13:13.690557 18402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.124:46807 every 8 connection(s)
I20260808 07:13:13.690637 18299 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
I20260808 07:13:13.691361 18403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:13.692814 18403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:13.700887 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 18299
I20260808 07:13:13.701004 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786173009519897-30937-0/minicluster-data/master-2/wal/instance
I20260808 07:13:13.701224 30937 external_mini_cluster.cc:1428] Setting key d258e8e907a4a8dbf3294f57379cbcc1
I20260808 07:13:13.701584 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.693451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.30.54.126:45657)
0808 07:13:13.693497 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.693499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.693682 (+ 183us) server_negotiation.cc:407] Connection header received
0808 07:13:13.694173 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.694175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.694187 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.694246 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.694874 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.695371 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.696342 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.696561 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.696649 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.698646 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.698648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.698648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.698657 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.700060 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.700484 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.700486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.700486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.700548 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.700797 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.700798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.700799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.700912 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.700994 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.701370 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.701483 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:13:13.701619 18406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.693402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45657 (local address 127.0.0.1:39750)
0808 07:13:13.693560 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.693602 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:13.694068 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:13.694310 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:13.694314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:13.694568 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:13.694760 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.694764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.695466 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.695468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.696251 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.696254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.696659 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.696660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.696789 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.697259 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:13.697268 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:13.698553 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:13.700164 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.700171 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.700173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.700394 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.700614 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.700615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.700615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.700723 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.701011 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:13.701018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:13.701183 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 07:13:13.701308 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:13:13.701473 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260808 07:13:13.702112 18403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
I20260808 07:13:13.708881 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.702441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.30.54.125:40195)
0808 07:13:13.702509 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.702511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.702521 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:13.702548 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.702549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.702560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.702622 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.702870 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.703397 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.704231 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.704524 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.704596 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.706242 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.706243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.706244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.706250 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.707727 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.708106 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.708107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.708108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.708168 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.708377 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.708377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.708378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.708482 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.708550 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.708704 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.708787 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:13:13.708882 18406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.702356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:40195 (local address 127.0.0.1:46402)
0808 07:13:13.702406 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.702417 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:13.702458 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:13.702625 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:13.702626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:13.702732 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:13.702778 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.702779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.703469 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.703470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.704160 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.704161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.704599 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.704600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.704648 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.704998 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:13.705002 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:13.706159 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:13.707818 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.707819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.707820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.708035 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.708239 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.708240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.708240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.708307 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.708555 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:13.708556 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:13.708598 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 07:13:13.708678 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:13:13.708794 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":44}
I20260808 07:13:13.709264 18403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:13:13.710732 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.701813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39314 (local address 127.30.54.124:46807)
0808 07:13:13.701969 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.701973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.702025 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:13:13.702111 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.702116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.702130 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.702216 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.703020 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.703875 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.704822 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.705229 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.705295 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.707801 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.707808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.707809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.707816 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.709577 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.709914 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.709917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.709917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.709978 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.710171 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.710172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.710173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.710312 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.710492 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.710537 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.710615 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:13.716908 18406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.709503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39330)
0808 07:13:13.709687 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.709698 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:13.709749 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:13.709884 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:13.709885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:13.710013 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:13.710037 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.710038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.710668 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.710669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.711439 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.711442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.711812 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.711813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.711861 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.712265 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:13.712270 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:13.713337 (+ 1067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:13.715948 (+ 2611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.715949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.715949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.716108 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.716309 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.716309 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.716310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.716365 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.716587 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:13.716588 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:13.716625 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 07:13:13.716706 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:13:13.716807 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260808 07:13:13.717108 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.709555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39330 (local address 127.30.54.124:46807)
0808 07:13:13.709701 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.709703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.709734 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:13:13.709804 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.709805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.709814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.709881 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.710108 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.710581 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.711512 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.711731 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.711811 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.713434 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.713435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.713435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.713441 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.715862 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.716189 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.716190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.716191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.716246 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.716432 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.716432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.716433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.716526 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.716594 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.716729 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.716877 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260808 07:13:13.718335 18403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed: Bootstrap starting.
I20260808 07:13:13.718377 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.711468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39754 (local address 127.30.54.126:45657)
0808 07:13:13.711503 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.711505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.711554 (+ 49us) server_negotiation.cc:407] Connection header received
0808 07:13:13.711625 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.711626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.711635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.711690 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.712127 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.712565 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.713255 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.713469 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.713539 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.715635 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.715636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.715637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.715644 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.717256 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.717634 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.717635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.717635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.717688 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.717883 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.717884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.717884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.717995 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.718176 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.718212 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.718284 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:13:13.719184 18403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:13.719476 18403 log.cc:824] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:13.720196 18403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed: No bootstrap required, opened a new log
I20260808 07:13:13.721750 18403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } }
I20260808 07:13:13.721890 18403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:13.721940 18403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29183a7931744372b9f1c19c3afd34ed, State: Initialized, Role: FOLLOWER
I20260808 07:13:13.722082 18403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } }
I20260808 07:13:13.722383 18415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.722440 18403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:13.722452 18415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
W20260808 07:13:13.724290 18434 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:13:13.724347 18434 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:13:13.726372 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.719038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46410 (local address 127.30.54.125:40195)
0808 07:13:13.719082 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.719084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.719093 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:13.719115 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.719116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.719124 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.719179 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.719648 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.720082 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.720762 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.720964 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.721033 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.723390 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.723391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.723391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.723398 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.725183 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.725577 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.725578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.725579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.725636 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.725840 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.725841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.725841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.725981 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.726171 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.726210 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.726286 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:13:13.726851 18180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } attempt: 2
I20260808 07:13:13.724802 18299 master_runner.cc:186] Error getting master registration for 127.30.54.126:45657: OK, instance_id { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" instance_seqno: 1786173193383051 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:13:13.727032 18299 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:13:13.727902 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
I20260808 07:13:13.734943 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.727325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39340 (local address 127.30.54.124:46807)
0808 07:13:13.727360 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.727362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.727371 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:13.727442 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.727443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.727455 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.727501 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.727784 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.728491 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.729654 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.729983 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.730053 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.732147 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.732148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.732149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.732156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.733811 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.734135 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.734137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.734137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.734190 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.734394 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.734395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.734395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.734526 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.734597 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.734752 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.734857 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:13:13.734951 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.727251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39340)
0808 07:13:13.727330 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.727339 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:13.727373 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:13.727518 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:13.727519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:13.727669 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:13.727703 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.727705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.728862 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.728863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.729557 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.729560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.730107 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.730108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.730163 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.730604 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:13.730609 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:13.732057 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:13.733889 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.733890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.733890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.734063 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.734262 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.734264 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.734264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.734324 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.734602 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:13.734603 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:13.734647 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:13:13.734727 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:13:13.734856 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 07:13:13.737129 18180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f: Bootstrap starting.
I20260808 07:13:13.737836 18180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:13.738163 18180 log.cc:824] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:13.739051 18180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f: No bootstrap required, opened a new log
I20260808 07:13:13.740176 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.729749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.30.54.125:40195)
0808 07:13:13.729781 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.729783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.730342 (+ 559us) server_negotiation.cc:407] Connection header received
0808 07:13:13.730440 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.730441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.730454 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.730485 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.731299 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.731982 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.733122 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.733428 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.733506 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.734987 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.734989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.734989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.734996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.736423 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.738737 (+ 2314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.738738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.738738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.738799 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.739088 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.739089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.739089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.739220 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.739755 (+ 535us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.740006 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.740089 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:13:13.741364 18180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } }
I20260808 07:13:13.741503 18180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:13.741539 18180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 217d5e3bb6cd4d518f5527b8c8bb1a5f, State: Initialized, Role: FOLLOWER
I20260808 07:13:13.741875 18180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } }
I20260808 07:13:13.742321 18180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:13.742656 18453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.742762 18453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:13.747640 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.730091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.30.54.124:46807)
0808 07:13:13.730159 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.730161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.740477 (+ 10316us) server_negotiation.cc:407] Connection header received
0808 07:13:13.740572 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.740573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.740583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.740611 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.741273 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.741986 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.742962 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.743327 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.743375 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.744800 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.744801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.744802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.744809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.746529 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.746885 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.746893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.746894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.746946 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.747158 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.747159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.747159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.747263 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.747372 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.747473 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.747546 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:13:13.746796 18074 master_runner.cc:186] Error getting master registration for 127.30.54.125:40195: OK, instance_id { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" instance_seqno: 1786173193535415 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:13:13.749577 18074 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:13:13.750039 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.729515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39760 (local address 127.30.54.126:45657)
0808 07:13:13.729550 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.729551 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.738976 (+ 9425us) server_negotiation.cc:407] Connection header received
0808 07:13:13.739080 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.739081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.739091 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.739122 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.739707 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.740428 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.741934 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.742237 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.742278 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.743773 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.743774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.743775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.743781 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.745331 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.745782 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.745783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.745783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.745842 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.746041 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.746042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.746042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.746155 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.746251 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.749867 (+ 3616us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.749945 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:13:13.751550 18469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:13:13.751605 18469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:13:13.767899 18290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } attempt: 3
Aug 08 07:13:13 dist-test-slave-znh6 krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
I20260808 07:13:13.776257 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.768194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46807 (local address 127.0.0.1:39352)
0808 07:13:13.768242 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:13.768252 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:13.768286 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:13.768427 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:13.768428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:13.768601 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:13.768650 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.768652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.769592 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.769593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.770488 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:13.770491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.770989 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.770990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:13.771052 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.771540 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:13.771545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:13.773051 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:13.775158 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.775159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.775160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.775380 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.775597 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:13.775598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:13.775598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:13.775665 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:13.775946 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:13.775946 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:13.775992 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:13:13.776065 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:13:13.776156 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:13:13.776268 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:13.768253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.30.54.124:46807)
0808 07:13:13.768289 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:13.768290 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:13.768299 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:13.768343 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:13.768344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:13.768352 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:13.768421 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:13.768727 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.769504 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.770561 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:13.770910 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:13.770982 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:13.773131 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:13.773132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:13.773132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:13.773139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:13.775081 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.775468 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.775469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.775470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.775528 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:13.775737 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:13.775737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:13.775738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:13.775876 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:13.775946 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:13.776097 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 07:13:13.776174 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:13:13.778007 18290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11: Bootstrap starting.
I20260808 07:13:13.778805 18290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:13.779153 18290 log.cc:824] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:13.780012 18290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11: No bootstrap required, opened a new log
I20260808 07:13:13.782349 18290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } }
I20260808 07:13:13.782495 18290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:13.782529 18290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 442fd9e386694bd3bb0d489d5fa52d11, State: Initialized, Role: FOLLOWER
I20260808 07:13:13.782645 18290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } }
I20260808 07:13:13.782974 18290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 07:13:13.783216 18221 tablet.cc:2369] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:13:13.784857 18496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:13:13.784976 18496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:13:13.786785 18188 master_runner.cc:263] No leader master found from master 442fd9e386694bd3bb0d489d5fa52d11
I20260808 07:13:13.786867 18188 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:13:13.782927 18474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.786926 18474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:13.866600 18453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:13.866742 18453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } }
I20260808 07:13:13.867015 18453 leader_election.cc:288] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 442fd9e386694bd3bb0d489d5fa52d11 (127.30.54.125:40195), 29183a7931744372b9f1c19c3afd34ed (127.30.54.124:46807)
I20260808 07:13:13.867506 18366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29183a7931744372b9f1c19c3afd34ed" is_pre_election: true
I20260808 07:13:13.867693 18366 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 217d5e3bb6cd4d518f5527b8c8bb1a5f in term 0.
I20260808 07:13:13.868006 18100 leader_election.cc:302] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 217d5e3bb6cd4d518f5527b8c8bb1a5f, 29183a7931744372b9f1c19c3afd34ed; no voters:
I20260808 07:13:13.868178 18453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:13.868235 18453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:13.868266 18453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:13.869302 18453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } }
I20260808 07:13:13.869436 18453 leader_election.cc:288] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [CANDIDATE]: Term 1 election: Requested vote from peers 442fd9e386694bd3bb0d489d5fa52d11 (127.30.54.125:40195), 29183a7931744372b9f1c19c3afd34ed (127.30.54.124:46807)
I20260808 07:13:13.869750 18366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29183a7931744372b9f1c19c3afd34ed"
I20260808 07:13:13.869836 18366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:13.870605 18255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442fd9e386694bd3bb0d489d5fa52d11" is_pre_election: true
I20260808 07:13:13.871074 18254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "442fd9e386694bd3bb0d489d5fa52d11"
I20260808 07:13:13.870774 18255 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 217d5e3bb6cd4d518f5527b8c8bb1a5f in term 0.
I20260808 07:13:13.872083 18366 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 217d5e3bb6cd4d518f5527b8c8bb1a5f in term 1.
I20260808 07:13:13.872277 18100 leader_election.cc:302] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 217d5e3bb6cd4d518f5527b8c8bb1a5f, 29183a7931744372b9f1c19c3afd34ed; no voters:
I20260808 07:13:13.873349 18453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:13.873486 18453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [term 1 LEADER]: Becoming Leader. State: Replica: 217d5e3bb6cd4d518f5527b8c8bb1a5f, State: Running, Role: LEADER
I20260808 07:13:13.873569 18453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } }
I20260808 07:13:13.873914 18453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 217d5e3bb6cd4d518f5527b8c8bb1a5f. Latest consensus state: current_term: 1 leader_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.873980 18453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:13.874222 18501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:13.874397 18501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:13.876371 18366 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [term 1 FOLLOWER]: Refusing update from remote peer 217d5e3bb6cd4d518f5527b8c8bb1a5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:13.876581 18255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:13.877370 18255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [term 1 FOLLOWER]: Refusing update from remote peer 217d5e3bb6cd4d518f5527b8c8bb1a5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:13.877632 18453 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:13.877936 18453 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:13.879659 18474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 217d5e3bb6cd4d518f5527b8c8bb1a5f. Latest consensus state: current_term: 1 leader_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.879748 18474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:13.880264 18415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 217d5e3bb6cd4d518f5527b8c8bb1a5f. Latest consensus state: current_term: 1 leader_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.880352 18415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:13:13.880868 18103 tablet.cc:2369] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:13:13.881901 18453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.881980 18453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:13.882282 18474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.882359 18474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:13.882570 18502 mvcc.cc:205] Tried to move back new op lower bound from 7316165402115067904 to 7316165402106642432. Current Snapshot: MvccSnapshot[applied={T|T < 7316165402115067904}]
I20260808 07:13:13.882987 18453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.883074 18453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:13.883401 18415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217d5e3bb6cd4d518f5527b8c8bb1a5f" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45657 } } peers { permanent_uuid: "442fd9e386694bd3bb0d489d5fa52d11" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 40195 } } peers { permanent_uuid: "29183a7931744372b9f1c19c3afd34ed" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46807 } } }
I20260808 07:13:13.883482 18415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:13:13.884202 18507 mvcc.cc:205] Tried to move back new op lower bound from 7316165402115067904 to 7316165402106642432. Current Snapshot: MvccSnapshot[applied={T|T < 7316165402115067904}]
I20260808 07:13:13.884694 18509 mvcc.cc:205] Tried to move back new op lower bound from 7316165402115067904 to 7316165402106642432. Current Snapshot: MvccSnapshot[applied={T|T < 7316165402115067904}]
I20260808 07:13:13.885210 18501 catalog_manager.cc:1347] Generated new cluster ID: 1ae44d908d754bb694fcc2195c02cd07
I20260808 07:13:13.885263 18501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:13:13.896288 18501 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:13:13.896812 18501 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:13.904086 18501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 217d5e3bb6cd4d518f5527b8c8bb1a5f: Generated new TSK 0
I20260808 07:13:13.904259 18501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:13:14.724773 18434 catalog_manager.cc:1259] Loaded cluster ID: 1ae44d908d754bb694fcc2195c02cd07
I20260808 07:13:14.724845 18434 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed: loading cluster ID for follower catalog manager: success
I20260808 07:13:14.725606 18434 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed: acquiring CA information for follower catalog manager: success
I20260808 07:13:14.725858 18434 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29183a7931744372b9f1c19c3afd34ed: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:13:14.785442 18496 catalog_manager.cc:1259] Loaded cluster ID: 1ae44d908d754bb694fcc2195c02cd07
I20260808 07:13:14.785518 18496 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11: loading cluster ID for follower catalog manager: success
I20260808 07:13:14.786302 18496 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11: acquiring CA information for follower catalog manager: success
I20260808 07:13:14.786556 18496 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 442fd9e386694bd3bb0d489d5fa52d11: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:13:14.956090 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 18074
I20260808 07:13:14.968353 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 18188
I20260808 07:13:14.975833 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 18299
2026-08-08T07:13:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3052 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:13:15 dist-test-slave-znh6 krb5kdc[18539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:13:15 dist-test-slave-znh6 krb5kdc[18539](info): set up 2 sockets
Aug 08 07:13:15 dist-test-slave-znh6 krb5kdc[18539](info): commencing operation
krb5kdc: starting...
W20260808 07:13:16.199934 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:13:16 dist-test-slave-znh6 krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:13:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:13:16.319868 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:44993
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:44993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:16.401211 18556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:16.401506 18556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:16.401546 18556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:16.403416 18556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:16.403470 18556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:16.403491 18556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:16.403509 18556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:16.403527 18556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:16.405742 18556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:44993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:44993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:16.406013 18556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:16.406260 18556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:16.411685 18561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:16.411710 18562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:16.411758 18556 server_base.cc:1034] running on GCE node
W20260808 07:13:16.411885 18564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:16.412174 18556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:16.412459 18556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:16.413638 18556 hybrid_clock.cc:648] HybridClock initialized: now 1786173196413628 us; error 34 us; skew 500 ppm
Aug 08 07:13:16 dist-test-slave-znh6 krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:16.419549 18556 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:13:16.419994 18556 webserver.cc:460] Webserver started at http://127.30.54.126:46445/ using document root <none> and password file <none>
I20260808 07:13:16.420223 18556 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:16.420290 18556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:16.420437 18556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:16.421481 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "96209199c2574be5be8cd8c56366e967"
format_stamp: "Formatted at 2026-08-08 07:13:16 on dist-test-slave-znh6"
server_key: "65f9fd53962a96aba549212cc4686e9f"
server_key_iv: "125a03b538335e876d8b9ab5159d2148"
server_key_version: "encryptionkey@0"
I20260808 07:13:16.421852 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "96209199c2574be5be8cd8c56366e967"
format_stamp: "Formatted at 2026-08-08 07:13:16 on dist-test-slave-znh6"
server_key: "65f9fd53962a96aba549212cc4686e9f"
server_key_iv: "125a03b538335e876d8b9ab5159d2148"
server_key_version: "encryptionkey@0"
I20260808 07:13:16.423249 18556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:13:16.423982 18574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:16.424167 18556 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:16.424265 18556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "96209199c2574be5be8cd8c56366e967"
format_stamp: "Formatted at 2026-08-08 07:13:16 on dist-test-slave-znh6"
server_key: "65f9fd53962a96aba549212cc4686e9f"
server_key_iv: "125a03b538335e876d8b9ab5159d2148"
server_key_version: "encryptionkey@0"
I20260808 07:13:16.424347 18556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:16.438941 18556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:16.441125 18556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:16.441288 18556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:16.445395 18556 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:44993
I20260808 07:13:16.445436 18651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:44993 every 8 connection(s)
I20260808 07:13:16.445806 18556 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:16.446692 18654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:16.448907 18654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967: Bootstrap starting.
I20260808 07:13:16.449442 18654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:16.449693 18654 log.cc:824] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:16.450438 18654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967: No bootstrap required, opened a new log
I20260808 07:13:16.451995 18654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96209199c2574be5be8cd8c56366e967" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44993 } }
I20260808 07:13:16.452127 18654 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:16.452150 18654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96209199c2574be5be8cd8c56366e967, State: Initialized, Role: FOLLOWER
I20260808 07:13:16.452292 18654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96209199c2574be5be8cd8c56366e967" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44993 } }
I20260808 07:13:16.452389 18654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:16.452414 18654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:16.452443 18654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:16.452922 18654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96209199c2574be5be8cd8c56366e967" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44993 } }
I20260808 07:13:16.453009 18654 leader_election.cc:302] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96209199c2574be5be8cd8c56366e967; no voters:
I20260808 07:13:16.453130 18654 leader_election.cc:288] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:13:16.453184 18658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:16.453363 18658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [term 1 LEADER]: Becoming Leader. State: Replica: 96209199c2574be5be8cd8c56366e967, State: Running, Role: LEADER
I20260808 07:13:16.453451 18658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96209199c2574be5be8cd8c56366e967" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44993 } }
I20260808 07:13:16.453452 18654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:16.453898 18660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96209199c2574be5be8cd8c56366e967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96209199c2574be5be8cd8c56366e967" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44993 } } }
I20260808 07:13:16.453992 18660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:16.454128 18661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96209199c2574be5be8cd8c56366e967. Latest consensus state: current_term: 1 leader_uuid: "96209199c2574be5be8cd8c56366e967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96209199c2574be5be8cd8c56366e967" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44993 } } }
I20260808 07:13:16.454255 18661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:16.454493 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 18556
I20260808 07:13:16.454591 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:13:16.454777 30937 external_mini_cluster.cc:1428] Setting key 4fd3d779bc00bc818f630b06ee4244b5
I20260808 07:13:16.454968 18666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:16.455519 18666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:16.457000 18666 catalog_manager.cc:1347] Generated new cluster ID: 392322e3ff2b45539ec69244a8925ba7
I20260808 07:13:16.457054 18666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:13:16 dist-test-slave-znh6 krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:16.477777 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:16.455441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48296 (local address 127.30.54.126:44993)
0808 07:13:16.466922 (+ 11481us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:16.466930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:16.466961 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:13:16.467056 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:16.467062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:16.467103 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:16.467222 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:16.468001 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:16.469446 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:16.470475 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:16.471256 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:16.471394 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:16.474208 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:16.474213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:16.474213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:16.474226 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:16.476169 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:16.476576 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:16.476579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:16.476579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:16.476681 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:16.476894 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:16.476895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:16.476895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:16.477174 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:16.477303 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:16.477426 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 07:13:16.477553 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11430,"thread_start_us":81,"threads_started":1}
I20260808 07:13:16.478845 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:13:16.480325 18666 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:13:16.480854 18666 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:16.488925 18666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96209199c2574be5be8cd8c56366e967: Generated new TSK 0
I20260808 07:13:16.489089 18666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:13:16.532956 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 18556
2026-08-08T07:13:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1557 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:13:16 dist-test-slave-znh6 krb5kdc[18695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:13:16 dist-test-slave-znh6 krb5kdc[18695](info): set up 2 sockets
Aug 08 07:13:16 dist-test-slave-znh6 krb5kdc[18695](info): commencing operation
krb5kdc: starting...
W20260808 07:13:17.775631 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:13:17 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:13:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:13:17.898414 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43295
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:43295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:17.978963 18720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:17.979197 18720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:17.979233 18720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:17.980370 18720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:17.980439 18720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:17.980464 18720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:17.980485 18720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:17.980538 18720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:17.981981 18720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:43295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:17.982249 18720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:17.982470 18720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:17.987417 18729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:17.987453 18726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:17.987480 18720 server_base.cc:1034] running on GCE node
W20260808 07:13:17.987419 18725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:17.987833 18720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:17.988111 18720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:17.989269 18720 hybrid_clock.cc:648] HybridClock initialized: now 1786173197989259 us; error 36 us; skew 500 ppm
Aug 08 07:13:17 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173197, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:17.991755 18720 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:13:17.992208 18720 webserver.cc:460] Webserver started at http://127.30.54.126:43183/ using document root <none> and password file <none>
I20260808 07:13:17.992442 18720 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:17.992517 18720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:17.992660 18720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:17.993670 18720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "3cc108d8cd42418f9922a5ae5fcd6db9"
format_stamp: "Formatted at 2026-08-08 07:13:17 on dist-test-slave-znh6"
server_key: "2b4877d800f18f17b05ed1a33f0383cc"
server_key_iv: "d903a14f210dada7cc2d3837f6bb721c"
server_key_version: "encryptionkey@0"
I20260808 07:13:17.994021 18720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "3cc108d8cd42418f9922a5ae5fcd6db9"
format_stamp: "Formatted at 2026-08-08 07:13:17 on dist-test-slave-znh6"
server_key: "2b4877d800f18f17b05ed1a33f0383cc"
server_key_iv: "d903a14f210dada7cc2d3837f6bb721c"
server_key_version: "encryptionkey@0"
I20260808 07:13:17.995389 18720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:13:17.996117 18737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:17.996281 18720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:17.996381 18720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "3cc108d8cd42418f9922a5ae5fcd6db9"
format_stamp: "Formatted at 2026-08-08 07:13:17 on dist-test-slave-znh6"
server_key: "2b4877d800f18f17b05ed1a33f0383cc"
server_key_iv: "d903a14f210dada7cc2d3837f6bb721c"
server_key_version: "encryptionkey@0"
I20260808 07:13:17.996482 18720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:18.010077 18720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:18.011969 18720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:18.012130 18720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:18.016264 18720 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:43295
I20260808 07:13:18.016304 18813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:43295 every 8 connection(s)
I20260808 07:13:18.016660 18720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:18.017628 18814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:18.020011 18814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Bootstrap starting.
I20260808 07:13:18.020591 18814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:18.020892 18814 log.cc:824] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:18.021587 18814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: No bootstrap required, opened a new log
I20260808 07:13:18.022954 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 18720
I20260808 07:13:18.023077 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:13:18.023272 30937 external_mini_cluster.cc:1428] Setting key 01625df22adba53d9a74fb891529a9e6
I20260808 07:13:18.023295 18814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:18.023488 18814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:18.023535 18814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cc108d8cd42418f9922a5ae5fcd6db9, State: Initialized, Role: FOLLOWER
I20260808 07:13:18.023691 18814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:18.023806 18814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:18.023849 18814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:18.023895 18814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:18.024444 18814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:18.024562 18814 leader_election.cc:302] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cc108d8cd42418f9922a5ae5fcd6db9; no voters:
I20260808 07:13:18.024744 18814 leader_election.cc:288] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:13:18.024770 18821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:18.024920 18821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 1 LEADER]: Becoming Leader. State: Replica: 3cc108d8cd42418f9922a5ae5fcd6db9, State: Running, Role: LEADER
I20260808 07:13:18.025008 18814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:18.025020 18821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:18.025382 18823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } } }
I20260808 07:13:18.025449 18826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cc108d8cd42418f9922a5ae5fcd6db9. Latest consensus state: current_term: 1 leader_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } } }
I20260808 07:13:18.025584 18826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:18.025795 18823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:18.025795 18837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:18.026388 18837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:18.028766 18837 catalog_manager.cc:1347] Generated new cluster ID: 4f94ed5bed2d49bba826c7474fd4fdeb
I20260808 07:13:18.028824 18837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:18.034142 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.023746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.30.54.126:43295)
0808 07:13:18.023961 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.023970 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.024003 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:13:18.024404 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.024410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.024454 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.024605 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:18.025408 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.026867 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.027757 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.028376 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.028490 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.030956 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:18.030960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:18.030960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:18.030971 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:18.032648 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.033021 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.033024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.033024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.033123 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.033330 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.033331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.033331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.033562 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:18.033743 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.033810 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.033951 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260808 07:13:18.040053 18837 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:13:18.040565 18837 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:18.046447 18837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Generated new TSK 0
I20260808 07:13:18.046623 18837 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:13:18.074401 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43295
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:18.157518 18856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:18.157716 18856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:18.157739 18856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:18.158905 18856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:18.158974 18856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:18.159024 18856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:13:18.160434 18856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:18.160619 18856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:18.160795 18856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:18.165706 18868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:18.165691 18863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:18.165691 18864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:18.165971 18856 server_base.cc:1034] running on GCE node
I20260808 07:13:18.166148 18856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:18.166364 18856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:18.167502 18856 hybrid_clock.cc:648] HybridClock initialized: now 1786173198167483 us; error 31 us; skew 500 ppm
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173198, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:18.169948 18856 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:13:18.170379 18856 webserver.cc:460] Webserver started at http://127.30.54.65:39819/ using document root <none> and password file <none>
I20260808 07:13:18.170562 18856 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:18.170603 18856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:18.170692 18856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:18.171705 18856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "eda48abf4ee44127a955bf490181b6d8"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "9d9d7b4fa7d70a5712bca1eaa4ceca29"
server_key_iv: "fa1717ca8fbd7c1769f778494cb449ef"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.171999 18856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "eda48abf4ee44127a955bf490181b6d8"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "9d9d7b4fa7d70a5712bca1eaa4ceca29"
server_key_iv: "fa1717ca8fbd7c1769f778494cb449ef"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.173209 18856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:13:18.173964 18876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.174145 18856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:18.174245 18856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "eda48abf4ee44127a955bf490181b6d8"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "9d9d7b4fa7d70a5712bca1eaa4ceca29"
server_key_iv: "fa1717ca8fbd7c1769f778494cb449ef"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.174316 18856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:18.188007 18856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:18.189754 18856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:18.189905 18856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:18.190119 18856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:18.190407 18856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:18.190459 18856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.190501 18856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:18.190542 18856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.196534 18856 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:32877
I20260808 07:13:18.196569 19054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:32877 every 8 connection(s)
I20260808 07:13:18.196857 18856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:18.199617 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 18856
I20260808 07:13:18.199726 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:13:18.199923 30937 external_mini_cluster.cc:1428] Setting key b7b751658dfd207d38968bc08ee4e003
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173198, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:18.206862 19060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.197488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:38517)
0808 07:13:18.197675 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.197708 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.198175 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:18.198424 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:18.198427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:18.199043 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:18.199268 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.199273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.200072 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.200073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.200982 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.200985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.201376 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.201377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.201496 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.201962 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:18.201969 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:18.203653 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:18.205527 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:18.205534 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:18.205536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:18.205744 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:18.205970 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:18.205970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:18.205971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:18.206073 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:18.206367 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:18.206372 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:18.206474 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 07:13:18.206557 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:13:18.206671 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260808 07:13:18.206871 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.197502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:38517 (local address 127.30.54.126:43295)
0808 07:13:18.197567 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.197570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.197789 (+ 219us) server_negotiation.cc:407] Connection header received
0808 07:13:18.198277 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.198278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.198293 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.198356 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:18.199359 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.199992 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.201066 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.201284 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.201355 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.203752 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:18.203753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:18.203753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:18.203762 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:18.205447 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.205823 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.205825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.205825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.205890 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.206141 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.206142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.206142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.206300 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:18.206379 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.206607 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.206704 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:13:18.207512 19055 heartbeater.cc:347] Connected to a master server at 127.30.54.126:43295
I20260808 07:13:18.207612 19055 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:18.207788 19055 heartbeater.cc:511] Master 127.30.54.126:43295 requested a full tablet report, sending...
I20260808 07:13:18.208283 18761 ts_manager.cc:194] Registered new tserver with Master: eda48abf4ee44127a955bf490181b6d8 (127.30.54.65:32877)
I20260808 07:13:18.208864 18761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:38517
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:13:18.229025 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43295
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:18.309067 19063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:18.309283 19063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:18.309319 19063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:18.310443 19063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:18.310505 19063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:18.310550 19063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:13:18.311985 19063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:18.312248 19063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:18.312463 19063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:18.317342 19082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:18.317428 19063 server_base.cc:1034] running on GCE node
W20260808 07:13:18.317308 19073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:18.317319 19078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:18.317715 19063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:18.317911 19063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:18.319065 19063 hybrid_clock.cc:648] HybridClock initialized: now 1786173198319047 us; error 44 us; skew 500 ppm
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173198, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:18.321501 19063 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:13:18.321892 19063 webserver.cc:460] Webserver started at http://127.30.54.66:37653/ using document root <none> and password file <none>
I20260808 07:13:18.322113 19063 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:18.322180 19063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:18.322321 19063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:18.323375 19063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "0d9eeac190ac495c8a5de45a4784db3c"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "02b50c261c362b76a31c973b13caad6e"
server_key_iv: "f44dab04320688a626af58277288e514"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.323747 19063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "0d9eeac190ac495c8a5de45a4784db3c"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "02b50c261c362b76a31c973b13caad6e"
server_key_iv: "f44dab04320688a626af58277288e514"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.325024 19063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:13:18.325784 19099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.325950 19063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:18.326051 19063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "0d9eeac190ac495c8a5de45a4784db3c"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "02b50c261c362b76a31c973b13caad6e"
server_key_iv: "f44dab04320688a626af58277288e514"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.326135 19063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:18.338243 19063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:18.340420 19063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:18.340575 19063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:18.340804 19063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:18.341096 19063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:18.341147 19063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.341187 19063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:18.341217 19063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.346971 19063 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:43641
I20260808 07:13:18.347005 19283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:43641 every 8 connection(s)
I20260808 07:13:18.347299 19063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173198, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:18.354048 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 19063
I20260808 07:13:18.354173 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:13:18.354401 30937 external_mini_cluster.cc:1428] Setting key 289f260c361c015c8936bd1139e08744
I20260808 07:13:18.356293 19286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.347933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:56577)
0808 07:13:18.348071 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.348110 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.348598 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:18.348831 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:18.348835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:18.349378 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:18.349552 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.349557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.350252 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.350253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.351110 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.351113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.351508 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.351509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.351642 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.352015 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:18.352022 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:18.353343 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:18.355004 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:18.355010 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:18.355011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:18.355276 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:18.355487 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:18.355487 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:18.355487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:18.355579 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:18.355844 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:18.355848 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:18.355968 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 07:13:18.356043 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:13:18.356153 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260808 07:13:18.356317 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.347965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:56577 (local address 127.30.54.126:43295)
0808 07:13:18.348008 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.348010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.348205 (+ 195us) server_negotiation.cc:407] Connection header received
0808 07:13:18.348700 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.348702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.348716 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.348769 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:18.349652 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.350160 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.351195 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.351413 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.351497 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.353426 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:18.353427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:18.353428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:18.353434 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:18.354935 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.355343 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.355344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.355345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.355415 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.355638 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.355639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.355639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.355776 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:18.355842 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.356084 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.356186 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:13:18.357121 19284 heartbeater.cc:347] Connected to a master server at 127.30.54.126:43295
I20260808 07:13:18.357221 19284 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:18.357411 19284 heartbeater.cc:511] Master 127.30.54.126:43295 requested a full tablet report, sending...
I20260808 07:13:18.357775 18761 ts_manager.cc:194] Registered new tserver with Master: 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641)
I20260808 07:13:18.358191 18761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:56577
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:13:18.380937 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43295
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:18.465075 19289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:18.465265 19289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:18.465288 19289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:18.466341 19289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:18.466378 19289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:18.466404 19289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:13:18.467775 19289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:18.467947 19289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:18.468113 19289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:18.473045 19298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:18.473090 19300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:18.473100 19297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:18.473374 19289 server_base.cc:1034] running on GCE node
I20260808 07:13:18.473593 19289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:18.473829 19289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:18.474995 19289 hybrid_clock.cc:648] HybridClock initialized: now 1786173198474987 us; error 37 us; skew 500 ppm
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173198, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:18.477464 19289 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:13:18.477844 19289 webserver.cc:460] Webserver started at http://127.30.54.67:36289/ using document root <none> and password file <none>
I20260808 07:13:18.478057 19289 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:18.478103 19289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:18.478263 19289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:18.479357 19289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "a8a9f0a9eab34317b8955c829d188e15"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "bc935862d5b54388847dd6b05a75cc6d"
server_key_iv: "e8aef9c194418e45c3c5c8da8626358f"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.479710 19289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "a8a9f0a9eab34317b8955c829d188e15"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "bc935862d5b54388847dd6b05a75cc6d"
server_key_iv: "e8aef9c194418e45c3c5c8da8626358f"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.481077 19289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:13:18.481781 19309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.481952 19289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:18.482038 19289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "a8a9f0a9eab34317b8955c829d188e15"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "bc935862d5b54388847dd6b05a75cc6d"
server_key_iv: "e8aef9c194418e45c3c5c8da8626358f"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.482108 19289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:18.498931 19289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:18.500793 19289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:18.500936 19289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:18.501138 19289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:18.501449 19289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:18.501503 19289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.501543 19289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:18.501569 19289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.507750 19289 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:35921
I20260808 07:13:18.507792 19487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:35921 every 8 connection(s)
I20260808 07:13:18.508090 19289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173198, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:18.515743 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 19289
I20260808 07:13:18.515866 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:13:18.516050 30937 external_mini_cluster.cc:1428] Setting key 96b97248ff9f69a2ae57fc9a705fe647
I20260808 07:13:18.516865 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.508789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:56627 (local address 127.30.54.126:43295)
0808 07:13:18.508831 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.508833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.509037 (+ 204us) server_negotiation.cc:407] Connection header received
0808 07:13:18.509490 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.509491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.509501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.509558 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:18.510429 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.510988 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.511920 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.512096 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.512146 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.514085 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:18.514087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:18.514087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:18.514094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:18.515472 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.515887 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.515888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.515889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.515973 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.516221 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.516222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.516222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.516369 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:18.516430 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.516695 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.516774 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:13:18.517079 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.508752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:56627)
0808 07:13:18.508917 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.508956 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.509412 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:18.509625 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:18.509628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:18.510172 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:18.510342 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.510346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.511069 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.511070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.511848 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.511850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.512166 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.512167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.512264 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.512668 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:18.512675 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:18.513991 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:18.515570 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:18.515578 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:18.515580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:18.515803 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:18.516035 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:18.516036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:18.516036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:18.516165 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:18.516437 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:18.516444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:18.516569 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 07:13:18.516671 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:13:18.516820 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260808 07:13:18.517580 19488 heartbeater.cc:347] Connected to a master server at 127.30.54.126:43295
I20260808 07:13:18.517683 19488 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:18.517853 19488 heartbeater.cc:511] Master 127.30.54.126:43295 requested a full tablet report, sending...
I20260808 07:13:18.518173 18761 ts_manager.cc:194] Registered new tserver with Master: a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921)
I20260808 07:13:18.518527 18761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:56627
I20260808 07:13:18.527751 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:18.534962 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.529110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50734 (local address 127.30.54.126:43295)
0808 07:13:18.529189 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.529191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.529293 (+ 102us) server_negotiation.cc:407] Connection header received
0808 07:13:18.529360 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.529361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.529371 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.529415 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:18.530086 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.530542 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.531329 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.531526 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.531599 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.532692 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:18.532693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:18.532694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:18.532700 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:18.533788 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.534168 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.534169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.534170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.534229 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.534429 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.534430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.534430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.534565 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:18.534665 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.534766 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.534865 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:13:18.536371 18761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:13:18.537835 18761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:13:18.543892 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.539806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:32877 (local address 127.0.0.1:37048)
0808 07:13:18.539944 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.539982 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.540047 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:18.540431 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:18.540434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:18.540437 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:18.540522 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.540524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.541571 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.541572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.542761 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.542763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.543581 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.543582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.543661 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.543674 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:13:18.543768 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:13:18.543822 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":52,"threads_started":1}
I20260808 07:13:18.543908 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.540053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37048 (local address 127.30.54.65:32877)
0808 07:13:18.540262 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.540266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.540275 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:18.540315 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.540317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.540328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.540422 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:18.540661 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.541470 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.542877 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.543497 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.543581 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.543638 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.543788 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.543826 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":105,"threads_started":1}
I20260808 07:13:18.544142 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.539838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:43641 (local address 127.0.0.1:39646)
0808 07:13:18.540149 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.540157 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.540196 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:18.540547 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:18.540548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:18.540551 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:18.540598 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.540601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.541886 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.541887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.542952 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.542955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.543914 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.543916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.543982 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.543986 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:18.544018 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 07:13:18.544056 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":2,"thread_start_us":59,"threads_started":1}
I20260808 07:13:18.544246 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.539838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:35921 (local address 127.0.0.1:48498)
0808 07:13:18.540524 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.540535 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.540587 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:18.540837 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:18.540838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:18.540840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:18.540885 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.540887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.542069 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.542070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.543055 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.543059 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.543983 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.543984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.544054 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.544057 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:13:18.544105 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:13:18.544138 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
I20260808 07:13:18.544268 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.540026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.30.54.67:35921)
0808 07:13:18.540212 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.540218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.540647 (+ 429us) server_negotiation.cc:407] Connection header received
0808 07:13:18.540690 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.540693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.540710 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.540780 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:18.540949 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.541978 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.543129 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.543839 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.543904 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.543954 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.544133 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.544176 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260808 07:13:18.544528 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.540221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.30.54.66:43641)
0808 07:13:18.540386 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.540390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.540400 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:18.540439 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.540441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.540453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.540534 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:18.540679 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.541782 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.543045 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.543799 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.544330 (+ 531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.544398 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.544424 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.544465 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260808 07:13:18.544914 19384 tablet_service.cc:1467] Processing CreateTablet for tablet bff8a981c6844e7aa1917057df6c7f00 (DEFAULT_TABLE table=test-table [id=7aa92c69ddeb4169825c0ba8a66eaaa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:18.545262 19384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff8a981c6844e7aa1917057df6c7f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:18.545168 18963 tablet_service.cc:1467] Processing CreateTablet for tablet bff8a981c6844e7aa1917057df6c7f00 (DEFAULT_TABLE table=test-table [id=7aa92c69ddeb4169825c0ba8a66eaaa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:18.545166 19174 tablet_service.cc:1467] Processing CreateTablet for tablet bff8a981c6844e7aa1917057df6c7f00 (DEFAULT_TABLE table=test-table [id=7aa92c69ddeb4169825c0ba8a66eaaa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:18.545476 19174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff8a981c6844e7aa1917057df6c7f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:18.545476 18963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff8a981c6844e7aa1917057df6c7f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:18.547966 19510 tablet_bootstrap.cc:492] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: Bootstrap starting.
I20260808 07:13:18.548454 19510 tablet_bootstrap.cc:654] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:18.548748 19511 tablet_bootstrap.cc:492] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: Bootstrap starting.
I20260808 07:13:18.549022 19510 log.cc:824] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:18.549371 19511 tablet_bootstrap.cc:654] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:18.549729 19510 tablet_bootstrap.cc:492] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: No bootstrap required, opened a new log
I20260808 07:13:18.549814 19510 ts_tablet_manager.cc:1397] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:13:18.549855 19511 log.cc:824] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:18.550184 19512 tablet_bootstrap.cc:492] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Bootstrap starting.
I20260808 07:13:18.550654 19511 tablet_bootstrap.cc:492] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: No bootstrap required, opened a new log
I20260808 07:13:18.550729 19511 ts_tablet_manager.cc:1397] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:13:18.550860 19512 tablet_bootstrap.cc:654] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:18.551409 19512 log.cc:824] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:18.551777 19510 raft_consensus.cc:348] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.551944 19510 raft_consensus.cc:374] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:18.551988 19510 raft_consensus.cc:729] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a9f0a9eab34317b8955c829d188e15, State: Initialized, Role: FOLLOWER
I20260808 07:13:18.552111 19510 consensus_queue.cc:260] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.552191 19512 tablet_bootstrap.cc:492] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: No bootstrap required, opened a new log
I20260808 07:13:18.552261 19512 ts_tablet_manager.cc:1397] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:13:18.552414 19488 heartbeater.cc:503] Master 127.30.54.126:43295 was elected leader, sending a full tablet report...
I20260808 07:13:18.552397 19510 ts_tablet_manager.cc:1428] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:13:18.552798 19511 raft_consensus.cc:348] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.552978 19511 raft_consensus.cc:374] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:18.553018 19511 raft_consensus.cc:729] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9eeac190ac495c8a5de45a4784db3c, State: Initialized, Role: FOLLOWER
I20260808 07:13:18.553155 19511 consensus_queue.cc:260] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.553426 19511 ts_tablet_manager.cc:1428] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:13:18.553444 19284 heartbeater.cc:503] Master 127.30.54.126:43295 was elected leader, sending a full tablet report...
I20260808 07:13:18.554270 19512 raft_consensus.cc:348] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.554431 19512 raft_consensus.cc:374] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:18.554486 19512 raft_consensus.cc:729] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda48abf4ee44127a955bf490181b6d8, State: Initialized, Role: FOLLOWER
I20260808 07:13:18.554591 19512 consensus_queue.cc:260] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.554881 19512 ts_tablet_manager.cc:1428] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:13:18.554908 19055 heartbeater.cc:503] Master 127.30.54.126:43295 was elected leader, sending a full tablet report...
W20260808 07:13:18.597868 19285 tablet.cc:2369] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:13:18.621641 19518 raft_consensus.cc:482] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:18.621758 19518 raft_consensus.cc:504] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.622225 19518 leader_election.cc:288] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eda48abf4ee44127a955bf490181b6d8 (127.30.54.65:32877), 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641)
I20260808 07:13:18.625234 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.622486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:35109 (local address 127.30.54.66:43641)
0808 07:13:18.622544 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.622546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.622681 (+ 135us) server_negotiation.cc:407] Connection header received
0808 07:13:18.622713 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.622714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.622723 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.622793 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:18.622988 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.623441 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.624402 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.624764 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.624960 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.624969 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.625121 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.625161 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:13:18.625242 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.622365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:32877 (local address 127.30.54.67:45431)
0808 07:13:18.622423 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.622438 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.622483 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:18.622701 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:18.622703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:18.622705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:18.622787 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.622789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.623524 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.623525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.624263 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.624265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.624972 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.624973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.625055 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.625058 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:13:18.625128 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:13:18.625166 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:13:18.625247 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.622517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:45431 (local address 127.30.54.65:32877)
0808 07:13:18.622569 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.622572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.622581 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:18.622609 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.622610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.622621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.622677 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:18.622903 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.623441 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.624360 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.624752 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.624808 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.624815 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.625144 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.625184 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:13:18.625249 19521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.622411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:43641 (local address 127.30.54.67:35109)
0808 07:13:18.622604 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.622614 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.622670 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:18.622802 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:18.622803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:18.622805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:18.622916 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.622918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.623540 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.623541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.624290 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.624292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.624991 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.624992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.625047 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.625051 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:18.625097 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:13:18.625153 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":126,"threads_started":1}
I20260808 07:13:18.625665 19203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff8a981c6844e7aa1917057df6c7f00" candidate_uuid: "a8a9f0a9eab34317b8955c829d188e15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9eeac190ac495c8a5de45a4784db3c" is_pre_election: true
I20260808 07:13:18.625703 18989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff8a981c6844e7aa1917057df6c7f00" candidate_uuid: "a8a9f0a9eab34317b8955c829d188e15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda48abf4ee44127a955bf490181b6d8" is_pre_election: true
I20260808 07:13:18.625830 18989 raft_consensus.cc:2454] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8a9f0a9eab34317b8955c829d188e15 in term 0.
I20260808 07:13:18.625826 19203 raft_consensus.cc:2454] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8a9f0a9eab34317b8955c829d188e15 in term 0.
I20260808 07:13:18.626060 19312 leader_election.cc:302] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9eeac190ac495c8a5de45a4784db3c, a8a9f0a9eab34317b8955c829d188e15; no voters:
I20260808 07:13:18.626238 19518 raft_consensus.cc:2781] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:18.626298 19518 raft_consensus.cc:482] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:18.626323 19518 raft_consensus.cc:3037] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:18.627206 19518 raft_consensus.cc:504] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.627346 19518 leader_election.cc:288] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [CANDIDATE]: Term 1 election: Requested vote from peers eda48abf4ee44127a955bf490181b6d8 (127.30.54.65:32877), 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641)
I20260808 07:13:18.627521 19203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff8a981c6844e7aa1917057df6c7f00" candidate_uuid: "a8a9f0a9eab34317b8955c829d188e15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d9eeac190ac495c8a5de45a4784db3c"
I20260808 07:13:18.627568 18989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff8a981c6844e7aa1917057df6c7f00" candidate_uuid: "a8a9f0a9eab34317b8955c829d188e15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda48abf4ee44127a955bf490181b6d8"
I20260808 07:13:18.627600 19203 raft_consensus.cc:3037] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:18.627671 18989 raft_consensus.cc:3037] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:18.628433 19203 raft_consensus.cc:2454] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8a9f0a9eab34317b8955c829d188e15 in term 1.
I20260808 07:13:18.628597 19312 leader_election.cc:302] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9eeac190ac495c8a5de45a4784db3c, a8a9f0a9eab34317b8955c829d188e15; no voters:
I20260808 07:13:18.628623 18989 raft_consensus.cc:2454] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8a9f0a9eab34317b8955c829d188e15 in term 1.
I20260808 07:13:18.628695 19518 raft_consensus.cc:2781] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:18.628914 19518 raft_consensus.cc:686] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 1 LEADER]: Becoming Leader. State: Replica: a8a9f0a9eab34317b8955c829d188e15, State: Running, Role: LEADER
I20260808 07:13:18.629056 19518 consensus_queue.cc:237] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.629914 18759 catalog_manager.cc:5495] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67). New cstate: current_term: 1 leader_uuid: "a8a9f0a9eab34317b8955c829d188e15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } health_report { overall_health: HEALTHY } } }
I20260808 07:13:18.658464 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.656224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.30.54.67:35921)
0808 07:13:18.656295 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.656298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.656309 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:13:18.656348 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.656349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.656370 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.656440 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:18.656572 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.657065 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.657768 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.657964 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.658014 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.658177 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:18.658274 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:18.658338 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.658373 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.658389 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:13:18.660332 19203 raft_consensus.cc:1261] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [term 1 FOLLOWER]: Refusing update from remote peer a8a9f0a9eab34317b8955c829d188e15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:18.660332 18989 raft_consensus.cc:1261] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Refusing update from remote peer a8a9f0a9eab34317b8955c829d188e15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:18.660693 19522 consensus_queue.cc:1035] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:18.660825 19518 consensus_queue.cc:1035] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:18.663838 19528 mvcc.cc:205] Tried to move back new op lower bound from 7316165421710082048 to 7316165421585305600. Current Snapshot: MvccSnapshot[applied={T|T < 7316165421710082048}]
I20260808 07:13:18.663998 19529 mvcc.cc:205] Tried to move back new op lower bound from 7316165421710082048 to 7316165421585305600. Current Snapshot: MvccSnapshot[applied={T|T < 7316165421710082048}]
I20260808 07:13:18.664047 19527 mvcc.cc:205] Tried to move back new op lower bound from 7316165421710082048 to 7316165421585305600. Current Snapshot: MvccSnapshot[applied={T|T < 7316165421710082048 or (T in {7316165421710082048})}]
I20260808 07:13:18.665261 18759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:18.665360 18759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:18.666091 18759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Sending DeleteTablet for 3 replicas of tablet bff8a981c6844e7aa1917057df6c7f00
I20260808 07:13:18.666415 19384 tablet_service.cc:1514] Processing DeleteTablet for tablet bff8a981c6844e7aa1917057df6c7f00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:18 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48498
I20260808 07:13:18.666467 18963 tablet_service.cc:1514] Processing DeleteTablet for tablet bff8a981c6844e7aa1917057df6c7f00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:18 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:37048
I20260808 07:13:18.666450 19174 tablet_service.cc:1514] Processing DeleteTablet for tablet bff8a981c6844e7aa1917057df6c7f00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:18 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:39646
I20260808 07:13:18.666692 19542 tablet_replica.cc:330] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: stopping tablet replica
I20260808 07:13:18.666741 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 18856
I20260808 07:13:18.666795 19542 raft_consensus.cc:2229] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:18.675372 19538 tablet_replica.cc:330] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: stopping tablet replica
I20260808 07:13:18.675429 19542 raft_consensus.cc:2258] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:18.675482 19538 raft_consensus.cc:2229] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:13:18.675613 19538 raft_consensus.cc:2258] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:18.675922 19542 ts_tablet_manager.cc:1905] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:18.676065 19538 ts_tablet_manager.cc:1905] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:18.677620 19538 ts_tablet_manager.cc:1918] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:18.677708 19538 log.cc:1196] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/bff8a981c6844e7aa1917057df6c7f00
I20260808 07:13:18.677968 19538 ts_tablet_manager.cc:1939] T bff8a981c6844e7aa1917057df6c7f00 P a8a9f0a9eab34317b8955c829d188e15: Deleting consensus metadata
I20260808 07:13:18.678032 19542 ts_tablet_manager.cc:1918] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:18.678113 19542 log.cc:1196] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/bff8a981c6844e7aa1917057df6c7f00
I20260808 07:13:18.678417 19542 ts_tablet_manager.cc:1939] T bff8a981c6844e7aa1917057df6c7f00 P 0d9eeac190ac495c8a5de45a4784db3c: Deleting consensus metadata
I20260808 07:13:18.678393 18740 catalog_manager.cc:4841] TS a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921): tablet bff8a981c6844e7aa1917057df6c7f00 (table test-table [id=7aa92c69ddeb4169825c0ba8a66eaaa0]) successfully deleted
I20260808 07:13:18.678831 18739 catalog_manager.cc:4841] TS 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641): tablet bff8a981c6844e7aa1917057df6c7f00 (table test-table [id=7aa92c69ddeb4169825c0ba8a66eaaa0]) successfully deleted
W20260808 07:13:18.679971 18738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:32877 (error 108)
I20260808 07:13:18.680476 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.680334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:32877 (local address 127.0.0.1:37062)
0808 07:13:18.680396 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.680443 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:32877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:13:18.680609 18738 catalog_manager.cc:4584] TS eda48abf4ee44127a955bf490181b6d8 (127.30.54.65:32877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bff8a981c6844e7aa1917057df6c7f00: Network error: Client connection negotiation failed: client connection to 127.30.54.65:32877: connect: Connection refused (error 111)
I20260808 07:13:18.680580 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 19063
I20260808 07:13:18.688988 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 19289
I20260808 07:13:18.701315 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 18720
I20260808 07:13:18.710459 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43295
--webserver_interface=127.30.54.126
--webserver_port=43183
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:43295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:18.793850 19544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:18.794075 19544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:18.794111 19544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:18.795264 19544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:18.795326 19544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:18.795351 19544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:18.795385 19544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:18.795416 19544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:18.796906 19544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:43295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=43183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:18.797175 19544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:18.797396 19544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:18.806561 19560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:18.806531 19556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:18.806577 19558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:18.806612 19544 server_base.cc:1034] running on GCE node
I20260808 07:13:18.806983 19544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:18.807276 19544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:18.808429 19544 hybrid_clock.cc:648] HybridClock initialized: now 1786173198808411 us; error 36 us; skew 500 ppm
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173198, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:18.810878 19544 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:13:18.811245 19544 webserver.cc:460] Webserver started at http://127.30.54.126:43183/ using document root <none> and password file <none>
I20260808 07:13:18.811450 19544 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:18.811499 19544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:18.812752 19544 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:18.813325 19569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.813499 19544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.813598 19544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "3cc108d8cd42418f9922a5ae5fcd6db9"
format_stamp: "Formatted at 2026-08-08 07:13:17 on dist-test-slave-znh6"
server_key: "2b4877d800f18f17b05ed1a33f0383cc"
server_key_iv: "d903a14f210dada7cc2d3837f6bb721c"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.813946 19544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:18.837623 19544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:18.839381 19544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:18.839536 19544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:18.843482 19544 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:43295
I20260808 07:13:18.843541 19647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:43295 every 8 connection(s)
I20260808 07:13:18.843926 19544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:18.845553 19648 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:13:18.845569 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 19544
I20260808 07:13:18.845873 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:32877
--local_ip_for_outbound_sockets=127.30.54.65
--tserver_master_addrs=127.30.54.126:43295
--webserver_port=39819
--webserver_interface=127.30.54.65
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:18.846302 19648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Bootstrap starting.
I20260808 07:13:18.854658 19648 log.cc:824] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:18.856537 19648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:18.856837 19648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Bootstrap complete.
I20260808 07:13:18.859154 19648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:18.859370 19648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cc108d8cd42418f9922a5ae5fcd6db9, State: Initialized, Role: FOLLOWER
I20260808 07:13:18.859539 19648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:18.859614 19648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:18.859678 19648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:18.859730 19648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:13:18.860711 19648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:18.860795 19648 leader_election.cc:302] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cc108d8cd42418f9922a5ae5fcd6db9; no voters:
I20260808 07:13:18.860982 19648 leader_election.cc:288] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:13:18.861028 19654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:13:18.861136 19654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 2 LEADER]: Becoming Leader. State: Replica: 3cc108d8cd42418f9922a5ae5fcd6db9, State: Running, Role: LEADER
I20260808 07:13:18.861253 19648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:18.861230 19654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:18.861508 19654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cc108d8cd42418f9922a5ae5fcd6db9. Latest consensus state: current_term: 2 leader_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } } }
I20260808 07:13:18.861605 19654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:18.861501 19655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } } }
I20260808 07:13:18.861898 19655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:18.862038 19660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:18.862798 19660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:18.863008 19660 catalog_manager.cc:1259] Loaded cluster ID: 4f94ed5bed2d49bba826c7474fd4fdeb
I20260808 07:13:18.863214 19660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:13:18.863505 19660 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 07:13:18.868379 19660 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 07:13:18.871330 19660 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:18.871508 19660 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Loaded TSK: 0
I20260808 07:13:18.871708 19660 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:13:18.931471 19651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:18.931666 19651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:18.931689 19651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:18.932826 19651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:18.932869 19651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:18.932895 19651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:13:18.934329 19651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:32877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=39819
--tserver_master_addrs=127.30.54.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:18.934558 19651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:18.934795 19651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:18.940729 19686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:18.940768 19691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:18.940754 19689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:18.941259 19651 server_base.cc:1034] running on GCE node
I20260808 07:13:18.941471 19651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:18.941696 19651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:18.942878 19651 hybrid_clock.cc:648] HybridClock initialized: now 1786173198942859 us; error 36 us; skew 500 ppm
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173198, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:18.945350 19651 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:13:18.945804 19651 webserver.cc:460] Webserver started at http://127.30.54.65:39819/ using document root <none> and password file <none>
I20260808 07:13:18.946034 19651 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:18.946113 19651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:18.947535 19651 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:18.948279 19700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:18.948470 19651 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:18.948552 19651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "eda48abf4ee44127a955bf490181b6d8"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "9d9d7b4fa7d70a5712bca1eaa4ceca29"
server_key_iv: "fa1717ca8fbd7c1769f778494cb449ef"
server_key_version: "encryptionkey@0"
I20260808 07:13:18.948871 19651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:18.970180 19651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:18.972384 19651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:18.972558 19651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:18.972800 19651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:18.973245 19719 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:13:18.975162 19651 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:13:18.975222 19651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:13:18.975281 19651 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:13:18.975883 19651 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:13:18.975939 19651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:18.976006 19719 tablet_bootstrap.cc:492] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Bootstrap starting.
I20260808 07:13:18.983343 19651 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:32877
I20260808 07:13:18.983505 19881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:32877 every 8 connection(s)
I20260808 07:13:18.983764 19651 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:18.986660 19719 log.cc:824] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:18.989120 19719 tablet_bootstrap.cc:492] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:18.989408 19719 tablet_bootstrap.cc:492] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Bootstrap complete.
I20260808 07:13:18.990600 19719 ts_tablet_manager.cc:1397] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
Aug 08 07:13:18 dist-test-slave-znh6 krb5kdc[18695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173198, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:18.991518 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 19651
I20260808 07:13:18.991833 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:43641
--local_ip_for_outbound_sockets=127.30.54.66
--tserver_master_addrs=127.30.54.126:43295
--webserver_port=37653
--webserver_interface=127.30.54.66
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:18.991922 19719 raft_consensus.cc:348] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.992282 19719 raft_consensus.cc:729] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda48abf4ee44127a955bf490181b6d8, State: Initialized, Role: FOLLOWER
I20260808 07:13:18.992420 19719 consensus_queue.cc:260] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } } peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } }
I20260808 07:13:18.992734 19719 ts_tablet_manager.cc:1428] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
I20260808 07:13:18.995117 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.984530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:34465)
0808 07:13:18.984854 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.984892 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.985352 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:18.985773 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:18.985776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:18.986049 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:18.986285 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.986289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.987675 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.987676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.988452 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:18.988454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.989073 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.989074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:18.989216 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.989775 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:18.989783 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:18.991228 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:18.993338 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:18.993344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:18.993346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:18.993678 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:18.993971 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:18.993972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:18.993972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:18.994074 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:18.994519 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:18.994523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:18.994624 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 07:13:18.994702 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:13:18.994860 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":96,"threads_started":1}
I20260808 07:13:18.995129 19886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.984599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:34465 (local address 127.30.54.126:43295)
0808 07:13:18.984767 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:18.984775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:18.984992 (+ 217us) server_negotiation.cc:407] Connection header received
0808 07:13:18.985557 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:18.985568 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:18.985611 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:18.985743 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:18.986409 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.987540 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.988566 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:18.988953 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:18.989088 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:18.991352 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:18.991356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:18.991356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:18.991368 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:18.993241 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.993770 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.993773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.993774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.993896 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:18.994140 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:18.994141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:18.994141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:18.994445 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:18.994535 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:18.994785 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 07:13:18.994961 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260808 07:13:18.995934 19882 heartbeater.cc:347] Connected to a master server at 127.30.54.126:43295
I20260808 07:13:18.996039 19882 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:18.996212 19882 heartbeater.cc:511] Master 127.30.54.126:43295 requested a full tablet report, sending...
I20260808 07:13:18.996892 19594 ts_manager.cc:194] Registered new tserver with Master: eda48abf4ee44127a955bf490181b6d8 (127.30.54.65:32877)
I20260808 07:13:18.997972 19594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:34465
I20260808 07:13:18.998744 19882 heartbeater.cc:503] Master 127.30.54.126:43295 was elected leader, sending a full tablet report...
I20260808 07:13:19.006173 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.997448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:32877 (local address 127.0.0.1:37076)
0808 07:13:18.997645 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:18.997673 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:18.997729 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:19.003111 (+ 5382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:19.003114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:19.003116 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:19.003162 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.003163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.004368 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.004369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.005121 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.005122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.005970 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.005971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.006026 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.006035 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:13:19.006082 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:13:19.006123 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":90,"threads_started":1}
I20260808 07:13:19.006210 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:18.997526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.30.54.65:32877)
0808 07:13:19.002947 (+ 5421us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.002952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.002964 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:13:19.002997 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.002999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.003013 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.003097 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:19.003233 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.004288 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.005198 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.005885 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.005955 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.006003 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.006111 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.006146 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5381,"thread_start_us":84,"threads_started":1}
I20260808 07:13:19.006594 19776 tablet_service.cc:1514] Processing DeleteTablet for tablet bff8a981c6844e7aa1917057df6c7f00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:18 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:37076
I20260808 07:13:19.007118 19893 tablet_replica.cc:330] stopping tablet replica
I20260808 07:13:19.007227 19893 raft_consensus.cc:2229] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:19.007308 19893 raft_consensus.cc:2258] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:19.007581 19893 ts_tablet_manager.cc:1905] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:19.009605 19893 ts_tablet_manager.cc:1918] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:19.009691 19893 log.cc:1196] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/bff8a981c6844e7aa1917057df6c7f00
I20260808 07:13:19.009905 19893 ts_tablet_manager.cc:1939] T bff8a981c6844e7aa1917057df6c7f00 P eda48abf4ee44127a955bf490181b6d8: Deleting consensus metadata
I20260808 07:13:19.010370 19570 catalog_manager.cc:4841] TS eda48abf4ee44127a955bf490181b6d8 (127.30.54.65:32877): tablet bff8a981c6844e7aa1917057df6c7f00 (table test-table [id=7aa92c69ddeb4169825c0ba8a66eaaa0]) successfully deleted
W20260808 07:13:19.075065 19890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:19.075250 19890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:19.075273 19890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:19.076314 19890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:19.076349 19890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:19.076375 19890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:13:19.077734 19890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:43641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=37653
--tserver_master_addrs=127.30.54.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:19.077924 19890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:19.078084 19890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:13:19.083034 19890 server_base.cc:1034] running on GCE node
W20260808 07:13:19.083024 19900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:19.083027 19899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:19.083177 19903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:19.083518 19890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:19.083752 19890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:19.084926 19890 hybrid_clock.cc:648] HybridClock initialized: now 1786173199084910 us; error 35 us; skew 500 ppm
Aug 08 07:13:19 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:19.087436 19890 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:13:19.087882 19890 webserver.cc:460] Webserver started at http://127.30.54.66:37653/ using document root <none> and password file <none>
I20260808 07:13:19.088110 19890 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:19.088191 19890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:19.089445 19890 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:19.089982 19911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.090149 19890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.090250 19890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "0d9eeac190ac495c8a5de45a4784db3c"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "02b50c261c362b76a31c973b13caad6e"
server_key_iv: "f44dab04320688a626af58277288e514"
server_key_version: "encryptionkey@0"
I20260808 07:13:19.090553 19890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:19.102324 19890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:19.104452 19890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:19.104600 19890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:19.104802 19890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:19.105085 19890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:19.105135 19890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.105168 19890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:19.105198 19890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.110931 19890 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:43641
I20260808 07:13:19.111020 20084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:43641 every 8 connection(s)
I20260808 07:13:19.111333 19890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:13:19 dist-test-slave-znh6 krb5kdc[18695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:19.118089 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 19890
I20260808 07:13:19.118408 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:35921
--local_ip_for_outbound_sockets=127.30.54.67
--tserver_master_addrs=127.30.54.126:43295
--webserver_port=36289
--webserver_interface=127.30.54.67
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:19.120322 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.111984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:48501)
0808 07:13:19.112149 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.112189 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:19.112626 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:19.112820 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:19.112822 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:19.113350 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:19.113560 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.113564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.114197 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.114198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.115046 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.115049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.115385 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.115386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.115488 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.115872 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:19.115880 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:19.117219 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:19.119089 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:19.119094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:19.119096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:19.119304 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:19.119522 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:19.119522 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:19.119523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:19.119621 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:19.119894 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:19.119898 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:19.119996 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 07:13:19.120074 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:13:19.120181 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260808 07:13:19.120360 19886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.112015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:48501 (local address 127.30.54.126:43295)
0808 07:13:19.112057 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.112060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.112268 (+ 208us) server_negotiation.cc:407] Connection header received
0808 07:13:19.112706 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.112707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.112716 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.112764 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:19.113633 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.114122 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.115121 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.115317 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.115371 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.117303 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:19.117304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:19.117305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:19.117311 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:19.119010 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:19.119378 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:19.119379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:19.119379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:19.119443 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:19.119680 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:19.119681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:19.119681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:19.119828 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:19.119899 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.120113 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.120225 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:13:19.120850 20085 heartbeater.cc:347] Connected to a master server at 127.30.54.126:43295
I20260808 07:13:19.120955 20085 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:19.121143 20085 heartbeater.cc:511] Master 127.30.54.126:43295 requested a full tablet report, sending...
I20260808 07:13:19.121515 19594 ts_manager.cc:194] Registered new tserver with Master: 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641)
I20260808 07:13:19.121876 19594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:48501
W20260808 07:13:19.201737 20088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:19.201920 20088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:19.201942 20088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:19.203172 20088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:19.203219 20088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:19.203281 20088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:13:19.204819 20088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:35921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=36289
--tserver_master_addrs=127.30.54.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:19.205045 20088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:19.205256 20088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:19.210072 20098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:19.210179 20101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:19.210103 20108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:19.210261 20088 server_base.cc:1034] running on GCE node
I20260808 07:13:19.210598 20088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:19.210945 20088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:19.212110 20088 hybrid_clock.cc:648] HybridClock initialized: now 1786173199212098 us; error 43 us; skew 500 ppm
Aug 08 07:13:19 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:19.214648 20088 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:13:19.215148 20088 webserver.cc:460] Webserver started at http://127.30.54.67:36289/ using document root <none> and password file <none>
I20260808 07:13:19.215374 20088 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:19.215457 20088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:19.216805 20088 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:19.217424 20118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.217602 20088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:19.217694 20088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "a8a9f0a9eab34317b8955c829d188e15"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "bc935862d5b54388847dd6b05a75cc6d"
server_key_iv: "e8aef9c194418e45c3c5c8da8626358f"
server_key_version: "encryptionkey@0"
I20260808 07:13:19.218007 20088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:19.242638 20088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:19.244413 20088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:19.244580 20088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:19.244808 20088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:19.245169 20088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:19.245223 20088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.245265 20088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:19.245292 20088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.251072 20088 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:35921
I20260808 07:13:19.251111 20295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:35921 every 8 connection(s)
I20260808 07:13:19.251462 20088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:13:19.255209 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 20088
Aug 08 07:13:19 dist-test-slave-znh6 krb5kdc[18695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:19.260650 19886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.252186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:59115 (local address 127.30.54.126:43295)
0808 07:13:19.252272 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.252274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.252429 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:13:19.252886 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.252887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.252898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.252942 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:19.253797 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.254296 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.255282 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.255548 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.255614 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.257820 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:19.257822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:19.257822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:19.257830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:19.259299 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:19.259726 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:19.259727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:19.259728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:19.259787 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:19.260033 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:19.260034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:19.260034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:19.260143 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:19.260216 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.260476 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.260558 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:13:19.260834 20299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.252152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:59115)
0808 07:13:19.252324 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.252358 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:19.252807 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:19.252999 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:19.253001 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:19.253548 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:19.253725 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.253729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.254372 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.254372 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.255168 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.255170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.255620 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.255622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.255776 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.256278 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:19.256289 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:19.257727 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:19.259407 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:19.259413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:19.259415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:19.259637 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:19.259860 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:19.259860 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:19.259861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:19.259962 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:19.260207 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:19.260213 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:19.260344 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 07:13:19.260455 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:13:19.260606 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260808 07:13:19.261304 20297 heartbeater.cc:347] Connected to a master server at 127.30.54.126:43295
I20260808 07:13:19.261410 20297 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:19.261610 20297 heartbeater.cc:511] Master 127.30.54.126:43295 requested a full tablet report, sending...
I20260808 07:13:19.262042 19594 ts_manager.cc:194] Registered new tserver with Master: a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921)
I20260808 07:13:19.262241 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.255592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50748 (local address 127.30.54.126:43295)
0808 07:13:19.255690 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.255691 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.255788 (+ 97us) server_negotiation.cc:407] Connection header received
0808 07:13:19.255853 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.255854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.255862 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.255918 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:19.256641 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.257150 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.257989 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.258190 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.258257 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.259424 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:19.259425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:19.259425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:19.259432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:19.260992 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:19.261413 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:19.261414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:19.261414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:19.261475 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:19.261695 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:19.261695 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:19.261696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:19.261807 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:19.262006 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.262056 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.262139 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":46,"threads_started":1}
I20260808 07:13:19.262540 19594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:59115
I20260808 07:13:19.262948 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:19.265625 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.263304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.30.54.126:43295)
0808 07:13:19.263347 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.263349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.263413 (+ 64us) server_negotiation.cc:407] Connection header received
0808 07:13:19.263496 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.263497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.263528 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.263609 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:19.263713 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.264162 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.264877 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.265086 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.265155 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.265321 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:19.265423 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:19.265511 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.265532 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.265554 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:13:19.265856 19593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:13:19.267735 19593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:13:19.269726 19776 tablet_service.cc:1467] Processing CreateTablet for tablet 4fa077480c45494bb6af9248a0690d2a (DEFAULT_TABLE table=test-table [id=7c37b03c29c94627a944595b5a1b62b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:19.269935 19776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fa077480c45494bb6af9248a0690d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:19.271579 19719 tablet_bootstrap.cc:492] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Bootstrap starting.
I20260808 07:13:19.271894 19719 tablet_bootstrap.cc:654] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:19.272471 19719 tablet_bootstrap.cc:492] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: No bootstrap required, opened a new log
I20260808 07:13:19.272543 19719 ts_tablet_manager.cc:1397] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:19.272655 19719 raft_consensus.cc:348] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.272728 19719 raft_consensus.cc:374] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:19.272780 19719 raft_consensus.cc:729] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda48abf4ee44127a955bf490181b6d8, State: Initialized, Role: FOLLOWER
I20260808 07:13:19.272835 19719 consensus_queue.cc:260] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.272927 19719 ts_tablet_manager.cc:1428] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.273403 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.269480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:35921 (local address 127.0.0.1:48520)
0808 07:13:19.269623 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.269633 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:19.269676 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:19.270010 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:19.270011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:19.270013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:19.270057 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.270059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.271245 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.271246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.272306 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.272310 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.273173 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.273174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.273234 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.273238 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:19.273297 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:13:19.273333 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":22,"thread_start_us":93,"threads_started":1}
I20260808 07:13:19.273423 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.269688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48520 (local address 127.30.54.67:35921)
0808 07:13:19.269844 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.269848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.269858 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:19.269894 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.269896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.269911 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.270010 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:19.270127 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.271127 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.272378 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.273102 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.273172 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.273230 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.273309 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.273349 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260808 07:13:19.274116 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.269415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:43641 (local address 127.0.0.1:39662)
0808 07:13:19.269447 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.269459 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:19.269501 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:19.270107 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:19.270109 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:19.270110 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:19.270137 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.270138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.271293 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.271293 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.272215 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.272219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.273920 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.273921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.273985 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.273988 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:13:19.274026 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 07:13:19.274062 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 07:13:19.274219 20196 tablet_service.cc:1467] Processing CreateTablet for tablet 4fa077480c45494bb6af9248a0690d2a (DEFAULT_TABLE table=test-table [id=7c37b03c29c94627a944595b5a1b62b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:19.274500 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.269725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39662 (local address 127.30.54.66:43641)
0808 07:13:19.269913 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.269917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.269926 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:19.269957 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.269958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.269987 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.270206 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:19.270255 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.271159 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.273030 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.273794 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.274268 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.274336 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.274362 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.274421 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260808 07:13:19.274598 20196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fa077480c45494bb6af9248a0690d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:19.275138 19992 tablet_service.cc:1467] Processing CreateTablet for tablet 4fa077480c45494bb6af9248a0690d2a (DEFAULT_TABLE table=test-table [id=7c37b03c29c94627a944595b5a1b62b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:19.275418 19992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fa077480c45494bb6af9248a0690d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:19.277555 20312 tablet_bootstrap.cc:492] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: Bootstrap starting.
I20260808 07:13:19.277961 20313 tablet_bootstrap.cc:492] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: Bootstrap starting.
I20260808 07:13:19.278229 20312 tablet_bootstrap.cc:654] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:19.278570 20313 tablet_bootstrap.cc:654] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:19.278764 20312 log.cc:824] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:19.279181 20313 log.cc:824] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:19.279618 20312 tablet_bootstrap.cc:492] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: No bootstrap required, opened a new log
I20260808 07:13:19.279673 20312 ts_tablet_manager.cc:1397] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:13:19.279865 20313 tablet_bootstrap.cc:492] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: No bootstrap required, opened a new log
I20260808 07:13:19.279939 20313 ts_tablet_manager.cc:1397] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:13:19.280926 20312 raft_consensus.cc:348] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.281054 20312 raft_consensus.cc:374] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:19.281077 20312 raft_consensus.cc:729] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a9f0a9eab34317b8955c829d188e15, State: Initialized, Role: FOLLOWER
I20260808 07:13:19.281188 20312 consensus_queue.cc:260] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.281338 20313 raft_consensus.cc:348] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.281477 20312 ts_tablet_manager.cc:1428] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:13:19.281500 20313 raft_consensus.cc:374] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:19.281534 20297 heartbeater.cc:503] Master 127.30.54.126:43295 was elected leader, sending a full tablet report...
I20260808 07:13:19.281538 20313 raft_consensus.cc:729] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9eeac190ac495c8a5de45a4784db3c, State: Initialized, Role: FOLLOWER
I20260808 07:13:19.281706 20313 consensus_queue.cc:260] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.281994 20313 ts_tablet_manager.cc:1428] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:13:19.282016 20085 heartbeater.cc:503] Master 127.30.54.126:43295 was elected leader, sending a full tablet report...
W20260808 07:13:19.361889 20086 tablet.cc:2369] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:13:19.484714 19883 tablet.cc:2369] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:13:19.493472 20320 raft_consensus.cc:482] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:19.493637 20320 raft_consensus.cc:504] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.494093 20320 leader_election.cc:288] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921), eda48abf4ee44127a955bf490181b6d8 (127.30.54.65:32877)
I20260808 07:13:19.497324 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.494368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:52901 (local address 127.30.54.67:35921)
0808 07:13:19.494417 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.494419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.494431 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:13:19.494544 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.494545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.494554 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.494619 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:19.494799 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.495348 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.496513 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.496957 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.497014 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.497021 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.497219 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.497255 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:13:19.497319 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.494292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:35921 (local address 127.30.54.66:52901)
0808 07:13:19.494373 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.494387 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:19.494442 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:19.494637 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:19.494638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:19.494640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:19.494718 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.494720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.495453 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.495454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.496445 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.496448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.497051 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.497052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.497140 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.497145 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:13:19.497202 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:13:19.497247 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:13:19.497514 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.494354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:32877 (local address 127.30.54.66:54873)
0808 07:13:19.494531 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.494540 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:19.494617 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:19.494764 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:19.494764 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:19.494766 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:19.494866 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.494867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.495582 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.495583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.496569 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:19.496572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.497225 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.497226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:19.497305 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.497309 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:19.497403 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:13:19.497444 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":105,"threads_started":1}
I20260808 07:13:19.497742 20227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa077480c45494bb6af9248a0690d2a" candidate_uuid: "0d9eeac190ac495c8a5de45a4784db3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a9f0a9eab34317b8955c829d188e15" is_pre_election: true
I20260808 07:13:19.497859 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.494439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:54873 (local address 127.30.54.65:32877)
0808 07:13:19.494484 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.494486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.494600 (+ 114us) server_negotiation.cc:407] Connection header received
0808 07:13:19.494674 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.494675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.494685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.494760 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:19.494955 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.495500 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.496676 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.497105 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.497200 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.497209 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.497749 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.497797 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:13:19.497920 20227 raft_consensus.cc:2454] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9eeac190ac495c8a5de45a4784db3c in term 0.
I20260808 07:13:19.498121 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa077480c45494bb6af9248a0690d2a" candidate_uuid: "0d9eeac190ac495c8a5de45a4784db3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda48abf4ee44127a955bf490181b6d8" is_pre_election: true
I20260808 07:13:19.498172 19914 leader_election.cc:302] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9eeac190ac495c8a5de45a4784db3c, a8a9f0a9eab34317b8955c829d188e15; no voters:
I20260808 07:13:19.498270 19813 raft_consensus.cc:2454] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9eeac190ac495c8a5de45a4784db3c in term 0.
I20260808 07:13:19.498298 20320 raft_consensus.cc:2781] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:19.498340 20320 raft_consensus.cc:482] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:19.498375 20320 raft_consensus.cc:3037] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:19.499121 20320 raft_consensus.cc:504] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.499262 20320 leader_election.cc:288] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [CANDIDATE]: Term 1 election: Requested vote from peers a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921), eda48abf4ee44127a955bf490181b6d8 (127.30.54.65:32877)
I20260808 07:13:19.499442 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa077480c45494bb6af9248a0690d2a" candidate_uuid: "0d9eeac190ac495c8a5de45a4784db3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda48abf4ee44127a955bf490181b6d8"
I20260808 07:13:19.499442 20227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa077480c45494bb6af9248a0690d2a" candidate_uuid: "0d9eeac190ac495c8a5de45a4784db3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a9f0a9eab34317b8955c829d188e15"
I20260808 07:13:19.499534 20227 raft_consensus.cc:3037] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:19.499534 19813 raft_consensus.cc:3037] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:19.500418 20227 raft_consensus.cc:2454] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9eeac190ac495c8a5de45a4784db3c in term 1.
I20260808 07:13:19.500419 19813 raft_consensus.cc:2454] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9eeac190ac495c8a5de45a4784db3c in term 1.
I20260808 07:13:19.500627 19912 leader_election.cc:302] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9eeac190ac495c8a5de45a4784db3c, eda48abf4ee44127a955bf490181b6d8; no voters:
I20260808 07:13:19.500730 20320 raft_consensus.cc:2781] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:19.500938 20320 raft_consensus.cc:686] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 1 LEADER]: Becoming Leader. State: Replica: 0d9eeac190ac495c8a5de45a4784db3c, State: Running, Role: LEADER
I20260808 07:13:19.501062 20320 consensus_queue.cc:237] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.501884 19594 catalog_manager.cc:5495] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66). New cstate: current_term: 1 leader_uuid: "0d9eeac190ac495c8a5de45a4784db3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } health_report { overall_health: UNKNOWN } } }
W20260808 07:13:19.502031 20298 tablet.cc:2369] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:13:19.556950 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.554702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39672 (local address 127.30.54.66:43641)
0808 07:13:19.554767 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:19.554769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:19.554777 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:13:19.554835 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:19.554836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:19.554853 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:19.554919 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:19.555040 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.555510 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.556207 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:19.556413 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:19.556470 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:19.556607 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:19.556703 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:19.556762 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:19.556803 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 07:13:19.556820 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:13:19.558799 20227 raft_consensus.cc:1261] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [term 1 FOLLOWER]: Refusing update from remote peer 0d9eeac190ac495c8a5de45a4784db3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:19.558799 19813 raft_consensus.cc:1261] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Refusing update from remote peer 0d9eeac190ac495c8a5de45a4784db3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:19.559229 20320 consensus_queue.cc:1035] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:19.559379 20322 consensus_queue.cc:1035] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:19.564157 19592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:19.564261 19592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:19.565027 19592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Sending DeleteTablet for 3 replicas of tablet 4fa077480c45494bb6af9248a0690d2a
I20260808 07:13:19.565366 19776 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fa077480c45494bb6af9248a0690d2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:19 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:37076
I20260808 07:13:19.565375 19992 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fa077480c45494bb6af9248a0690d2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:19 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:39662
I20260808 07:13:19.565683 20342 tablet_replica.cc:330] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: stopping tablet replica
I20260808 07:13:19.565706 20196 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fa077480c45494bb6af9248a0690d2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:19 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48520
I20260808 07:13:19.565747 20342 raft_consensus.cc:2229] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:19.565752 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 19651
I20260808 07:13:19.575589 20344 tablet_replica.cc:330] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: stopping tablet replica
I20260808 07:13:19.575717 20344 raft_consensus.cc:2229] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:19.575815 20344 raft_consensus.cc:2258] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:19.578230 20344 ts_tablet_manager.cc:1905] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:19.580310 20344 ts_tablet_manager.cc:1918] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:19.580411 20344 log.cc:1196] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/4fa077480c45494bb6af9248a0690d2a
I20260808 07:13:19.580822 20344 ts_tablet_manager.cc:1939] T 4fa077480c45494bb6af9248a0690d2a P a8a9f0a9eab34317b8955c829d188e15: Deleting consensus metadata
I20260808 07:13:19.587448 19572 catalog_manager.cc:4841] TS a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921): tablet 4fa077480c45494bb6af9248a0690d2a (table test-table [id=7c37b03c29c94627a944595b5a1b62b0]) successfully deleted
I20260808 07:13:19.587697 20343 tablet_replica.cc:330] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: stopping tablet replica
I20260808 07:13:19.587849 20343 raft_consensus.cc:2229] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:13:19.587990 20343 raft_consensus.cc:2258] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:19.588981 20343 ts_tablet_manager.cc:1905] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:19.591096 20343 ts_tablet_manager.cc:1918] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:19.591180 20343 log.cc:1196] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/4fa077480c45494bb6af9248a0690d2a
I20260808 07:13:19.591450 20343 ts_tablet_manager.cc:1939] T 4fa077480c45494bb6af9248a0690d2a P 0d9eeac190ac495c8a5de45a4784db3c: Deleting consensus metadata
I20260808 07:13:19.591924 19571 catalog_manager.cc:4841] TS 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641): tablet 4fa077480c45494bb6af9248a0690d2a (table test-table [id=7c37b03c29c94627a944595b5a1b62b0]) successfully deleted
W20260808 07:13:19.593029 19570 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:32877 (error 108)
I20260808 07:13:19.593492 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 19890
I20260808 07:13:19.593581 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.593442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:32877 (local address 127.0.0.1:37088)
0808 07:13:19.593493 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.593541 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:32877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 07:13:19.598096 19570 catalog_manager.cc:4584] TS eda48abf4ee44127a955bf490181b6d8 (127.30.54.65:32877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4fa077480c45494bb6af9248a0690d2a: Network error: Client connection negotiation failed: client connection to 127.30.54.65:32877: connect: Connection refused (error 111)
I20260808 07:13:19.603941 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 20088
I20260808 07:13:19.613538 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 19544
I20260808 07:13:19.623829 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43295
--webserver_interface=127.30.54.126
--webserver_port=43183
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:43295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:19.704547 20346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:19.704735 20346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:19.704756 20346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:19.705875 20346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:19.705915 20346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:19.705930 20346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:19.705971 20346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:19.706002 20346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:19.707436 20346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:43295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=43183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:19.707688 20346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:19.707916 20346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:19.712679 20356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:19.712713 20354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:19.712869 20346 server_base.cc:1034] running on GCE node
W20260808 07:13:19.712910 20353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:19.713137 20346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:19.713361 20346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:19.714545 20346 hybrid_clock.cc:648] HybridClock initialized: now 1786173199714534 us; error 26 us; skew 500 ppm
Aug 08 07:13:19 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:19.717017 20346 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:13:19.717504 20346 webserver.cc:460] Webserver started at http://127.30.54.126:43183/ using document root <none> and password file <none>
I20260808 07:13:19.717722 20346 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:19.717792 20346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:19.719136 20346 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:19.719823 20370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.719988 20346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:19.720093 20346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "3cc108d8cd42418f9922a5ae5fcd6db9"
format_stamp: "Formatted at 2026-08-08 07:13:17 on dist-test-slave-znh6"
server_key: "2b4877d800f18f17b05ed1a33f0383cc"
server_key_iv: "d903a14f210dada7cc2d3837f6bb721c"
server_key_version: "encryptionkey@0"
I20260808 07:13:19.720423 20346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:19.743314 20346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:19.745036 20346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:19.745199 20346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:19.749477 20346 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:43295
I20260808 07:13:19.749518 20449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:43295 every 8 connection(s)
I20260808 07:13:19.749904 20346 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:19.751736 20450 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:13:19.752753 20450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Bootstrap starting.
I20260808 07:13:19.759274 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 20346
I20260808 07:13:19.759580 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:32877
--local_ip_for_outbound_sockets=127.30.54.65
--tserver_master_addrs=127.30.54.126:43295
--webserver_port=39819
--webserver_interface=127.30.54.65
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:19.761670 20450 log.cc:824] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:19.764408 20450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:19.764721 20450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9: Bootstrap complete.
I20260808 07:13:19.767051 20450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:19.767274 20450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cc108d8cd42418f9922a5ae5fcd6db9, State: Initialized, Role: FOLLOWER
I20260808 07:13:19.767439 20450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:19.767503 20450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:19.767566 20450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:19.767611 20450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:13:19.768605 20450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:19.768693 20450 leader_election.cc:302] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cc108d8cd42418f9922a5ae5fcd6db9; no voters:
I20260808 07:13:19.768894 20450 leader_election.cc:288] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:13:19.768931 20457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:13:19.769049 20457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [term 3 LEADER]: Becoming Leader. State: Replica: 3cc108d8cd42418f9922a5ae5fcd6db9, State: Running, Role: LEADER
I20260808 07:13:19.769147 20457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } }
I20260808 07:13:19.769186 20450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:19.769411 20458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } } }
I20260808 07:13:19.769691 20458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:19.769599 20461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cc108d8cd42418f9922a5ae5fcd6db9. Latest consensus state: current_term: 3 leader_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc108d8cd42418f9922a5ae5fcd6db9" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43295 } } }
I20260808 07:13:19.769972 20470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:19.769985 20461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc108d8cd42418f9922a5ae5fcd6db9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:19.770807 20470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:19.770946 20470 catalog_manager.cc:1259] Loaded cluster ID: 4f94ed5bed2d49bba826c7474fd4fdeb
I20260808 07:13:19.770982 20470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 07:13:19.771268 20470 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc43ecc29ad google::LogMessage::Fail() at ??:0
@ 0x7fc43ecc588a google::LogMessage::SendToLog() at ??:0
@ 0x7fc43ecc24de google::LogMessage::Flush() at ??:0
@ 0x7fc43ecc3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc446be30f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc446bec8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc440b900d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc440b85954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc4419e96db start_thread at ??:0
@ 0x7fc43d7c061f clone at ??:0
W20260808 07:13:19.851501 20454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:19.851713 20454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:19.851752 20454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:19.852844 20454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:19.852905 20454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:19.852948 20454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:13:19.854387 20454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:32877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=39819
--tserver_master_addrs=127.30.54.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:19.854648 20454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:19.854918 20454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:13:19.861135 20454 server_base.cc:1034] running on GCE node
W20260808 07:13:19.861265 20510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:19.861135 20505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:19.861367 20503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:19.861625 20454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:19.861857 20454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:19.863023 20454 hybrid_clock.cc:648] HybridClock initialized: now 1786173199863012 us; error 36 us; skew 500 ppm
Aug 08 07:13:19 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:19.865501 20454 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:13:19.865998 20454 webserver.cc:460] Webserver started at http://127.30.54.65:39819/ using document root <none> and password file <none>
I20260808 07:13:19.866218 20454 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:19.866264 20454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:19.867604 20454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:19.868264 20521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.868433 20454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:19.868528 20454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "eda48abf4ee44127a955bf490181b6d8"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "9d9d7b4fa7d70a5712bca1eaa4ceca29"
server_key_iv: "fa1717ca8fbd7c1769f778494cb449ef"
server_key_version: "encryptionkey@0"
I20260808 07:13:19.868825 20454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:19.882268 20454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:19.884387 20454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:19.884522 20454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:19.884763 20454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:19.885202 20530 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:13:19.886574 20454 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:13:19.886624 20454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:13:19.886665 20454 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:13:19.887315 20454 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:13:19.887362 20454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:19.887415 20530 tablet_bootstrap.cc:492] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Bootstrap starting.
I20260808 07:13:19.895215 20454 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:32877
I20260808 07:13:19.895354 20687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:32877 every 8 connection(s)
I20260808 07:13:19.895700 20454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:19.895831 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 20454
I20260808 07:13:19.896133 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:43641
--local_ip_for_outbound_sockets=127.30.54.66
--tserver_master_addrs=127.30.54.126:43295
--webserver_port=37653
--webserver_interface=127.30.54.66
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:19.898419 20530 log.cc:824] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:19.901137 20530 tablet_bootstrap.cc:492] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:19.901465 20530 tablet_bootstrap.cc:492] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Bootstrap complete.
I20260808 07:13:19.902695 20530 ts_tablet_manager.cc:1397] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260808 07:13:19.904209 20530 raft_consensus.cc:348] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.904524 20530 raft_consensus.cc:729] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda48abf4ee44127a955bf490181b6d8, State: Initialized, Role: FOLLOWER
I20260808 07:13:19.904663 20530 consensus_queue.cc:260] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:19.904923 20530 ts_tablet_manager.cc:1428] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
W20260808 07:13:19.957415 20692 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:13:19.898383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:41993)
0808 07:13:19.898952 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.899000 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:19.899523 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:19.957214 (+ 57691us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":92,"threads_started":1}
W20260808 07:13:19.957708 20688 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:43295 (0 consecutive failures): Network error: Failed to ping master at 127.30.54.126:43295: Client connection negotiation failed: client connection to 127.30.54.126:43295: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:13:19.958084 20692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.957955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:40573)
0808 07:13:19.958011 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.958059 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:13:19.958434 20692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:19.958332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:34927)
0808 07:13:19.958382 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:19.958407 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:13:19.958573 20688 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:13:19.979947 20691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:19.980142 20691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:19.980180 20691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:19.981289 20691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:19.981354 20691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:19.981398 20691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:13:19.982851 20691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:43641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=37653
--tserver_master_addrs=127.30.54.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:19.983104 20691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:19.983337 20691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:19.988161 20708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:19.988175 20691 server_base.cc:1034] running on GCE node
W20260808 07:13:19.988255 20704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:19.988169 20703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:19.988528 20691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:19.988752 20691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:19.989924 20691 hybrid_clock.cc:648] HybridClock initialized: now 1786173199989921 us; error 38 us; skew 500 ppm
Aug 08 07:13:19 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:19.992441 20691 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:13:19.992859 20691 webserver.cc:460] Webserver started at http://127.30.54.66:37653/ using document root <none> and password file <none>
I20260808 07:13:19.993046 20691 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:19.993088 20691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:19.994313 20691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:19.994905 20723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:19.995085 20691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:19.995168 20691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "0d9eeac190ac495c8a5de45a4784db3c"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "02b50c261c362b76a31c973b13caad6e"
server_key_iv: "f44dab04320688a626af58277288e514"
server_key_version: "encryptionkey@0"
I20260808 07:13:19.995437 20691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:20.003865 20691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:20.006021 20691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:20.006125 20691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:20.006278 20691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:20.006497 20691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:20.006529 20691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:20.006549 20691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:20.006561 20691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:20.012341 20691 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:43641
I20260808 07:13:20.012442 20891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:43641 every 8 connection(s)
I20260808 07:13:20.012727 20691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:13:20.013650 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:20.013302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:48471)
0808 07:13:20.013448 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:20.013524 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
W20260808 07:13:20.014032 20892 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:43295 (0 consecutive failures): Network error: Failed to ping master at 127.30.54.126:43295: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
I20260808 07:13:20.014299 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:20.014204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:56167)
0808 07:13:20.014252 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:20.014276 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:13:20.014508 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 20691
I20260808 07:13:20.014638 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:20.014551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:47571)
0808 07:13:20.014595 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:20.014618 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:13:20.014747 20892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:13:20.014868 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:35921
--local_ip_for_outbound_sockets=127.30.54.67
--tserver_master_addrs=127.30.54.126:43295
--webserver_port=36289
--webserver_interface=127.30.54.67
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:20.096753 20896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:20.096935 20896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:20.096976 20896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:20.098093 20896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:20.098130 20896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:20.098156 20896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:13:20.099574 20896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:35921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=36289
--tserver_master_addrs=127.30.54.126:43295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:20.099784 20896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:20.099974 20896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:20.104914 20909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:20.104965 20904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:20.105130 20896 server_base.cc:1034] running on GCE node
W20260808 07:13:20.104913 20902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:20.105409 20896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:20.105604 20896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:20.106741 20896 hybrid_clock.cc:648] HybridClock initialized: now 1786173200106707 us; error 52 us; skew 500 ppm
Aug 08 07:13:20 dist-test-slave-znh6 krb5kdc[18695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173200, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:20.109306 20896 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:13:20.109771 20896 webserver.cc:460] Webserver started at http://127.30.54.67:36289/ using document root <none> and password file <none>
I20260808 07:13:20.109987 20896 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:20.110036 20896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:20.111372 20896 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:13:20.111963 20917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:20.112143 20896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:13:20.112234 20896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "a8a9f0a9eab34317b8955c829d188e15"
format_stamp: "Formatted at 2026-08-08 07:13:18 on dist-test-slave-znh6"
server_key: "bc935862d5b54388847dd6b05a75cc6d"
server_key_iv: "e8aef9c194418e45c3c5c8da8626358f"
server_key_version: "encryptionkey@0"
I20260808 07:13:20.112543 20896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:20.132699 20896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:20.134523 20896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:20.134665 20896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:20.134938 20896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:20.135242 20896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:20.135298 20896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:20.135340 20896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:20.135367 20896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:20.141419 20896 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:35921
I20260808 07:13:20.141472 21092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:35921 every 8 connection(s)
I20260808 07:13:20.141795 20896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:13:20.142778 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:20.142432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:38903)
0808 07:13:20.142576 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:20.142653 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
W20260808 07:13:20.143193 21093 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:43295 (0 consecutive failures): Network error: Failed to ping master at 127.30.54.126:43295: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
I20260808 07:13:20.143507 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:20.143416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:55049)
0808 07:13:20.143462 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:20.143488 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:13:20.143836 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:20.143758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:58943)
0808 07:13:20.143799 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:20.143821 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:13:20.143980 21093 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:13:20.150480 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 20896
I20260808 07:13:20.959193 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:20.958949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:38979)
0808 07:13:20.959085 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:20.959120 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260808 07:13:21.015370 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:21.015185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:33131)
0808 07:13:21.015296 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:21.015326 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260808 07:13:21.144485 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:21.144270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:58577)
0808 07:13:21.144401 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:21.144434 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260808 07:13:21.237387 21102 raft_consensus.cc:482] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:21.237511 21102 raft_consensus.cc:504] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:21.237908 21102 leader_election.cc:288] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641), a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921)
Aug 08 07:13:21 dist-test-slave-znh6 krb5kdc[18695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.67@KRBTEST.COM
Aug 08 07:13:21 dist-test-slave-znh6 krb5kdc[18695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.66@KRBTEST.COM
I20260808 07:13:21.251068 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:21.238159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:52115 (local address 127.30.54.66:43641)
0808 07:13:21.238271 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:21.238282 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:21.238320 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:13:21.238951 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:21.238954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:21.239017 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:21.239176 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:21.240244 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:21.241645 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:21.243225 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:21.243877 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:21.244017 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:21.247485 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:21.247488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:21.247489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:21.247500 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:21.249365 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:21.249717 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:21.249720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:21.249720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:21.249810 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:21.250028 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:21.250029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:21.250030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:21.250282 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:21.250375 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:21.250785 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 07:13:21.250949 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":56,"threads_started":1}
I20260808 07:13:21.251066 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:21.238204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:47601 (local address 127.30.54.67:35921)
0808 07:13:21.238358 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:21.238365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:21.238392 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:13:21.238950 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:21.238954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:21.239017 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:21.239176 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:21.240244 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:21.241646 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:21.243225 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:21.243877 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:21.244001 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:21.247045 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:21.247050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:21.247050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:21.247062 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:21.249102 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:21.249550 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:21.249552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:21.249552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:21.249639 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:21.249949 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:21.249950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:21.249950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:21.250283 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:21.250375 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:21.250785 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 07:13:21.250949 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260808 07:13:21.251073 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:21.238031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:43641 (local address 127.30.54.65:52115)
0808 07:13:21.238087 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:21.238098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:21.238156 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:21.239255 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:21.239258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:21.239795 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:21.240100 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:21.240107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:21.241787 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:21.241788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:21.243127 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:21.243130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:21.243999 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:21.243999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:21.244173 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:21.244838 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:21.244861 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:21.247382 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:21.249453 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:21.249454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:21.249455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:21.249638 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:21.249879 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:21.249879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:21.249880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:21.249950 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:21.250408 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:21.250409 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:21.250570 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 07:13:21.250711 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 07:13:21.250943 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":86}
I20260808 07:13:21.251305 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:21.238086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:35921 (local address 127.30.54.65:47601)
0808 07:13:21.238208 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:21.238219 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:21.238276 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:21.239255 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:21.239258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:21.239795 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:21.240100 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:21.240107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:21.241777 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:21.241778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:21.243127 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:21.243130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:21.243997 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:21.243999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:21.244173 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:21.244853 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:21.244861 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:21.246922 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:21.249192 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:21.249199 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:21.249201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:21.249459 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:21.249736 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:21.249737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:21.249737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:21.249859 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:21.250391 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:21.250399 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:21.250570 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 07:13:21.250711 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 07:13:21.251005 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":268,"thread_start_us":68,"threads_started":1}
W20260808 07:13:21.252014 20524 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
W20260808 07:13:21.252454 20525 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
I20260808 07:13:21.252518 20525 leader_election.cc:302] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eda48abf4ee44127a955bf490181b6d8; no voters: 0d9eeac190ac495c8a5de45a4784db3c, a8a9f0a9eab34317b8955c829d188e15
I20260808 07:13:21.252641 21102 raft_consensus.cc:2726] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:13:21.959882 21109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:21.959644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:44141)
0808 07:13:21.959795 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:21.959833 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260808 07:13:22.016032 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:22.015849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:41865)
0808 07:13:22.015965 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:22.015998 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260808 07:13:22.145144 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:22.144918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:37331)
0808 07:13:22.145073 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:22.145106 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260808 07:13:22.960520 21116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:22.960316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:46741)
0808 07:13:22.960441 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:22.960472 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260808 07:13:23.016600 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:23.016412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:56941)
0808 07:13:23.016530 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:23.016564 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260808 07:13:23.145819 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:23.145572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:57639)
0808 07:13:23.145727 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:23.145772 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260808 07:13:23.479014 21121 raft_consensus.cc:482] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:23.479110 21121 raft_consensus.cc:504] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:23.479326 21121 leader_election.cc:288] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641), a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921)
W20260808 07:13:23.479784 20525 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
W20260808 07:13:23.479880 20524 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
I20260808 07:13:23.479940 20524 leader_election.cc:302] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eda48abf4ee44127a955bf490181b6d8; no voters: 0d9eeac190ac495c8a5de45a4784db3c, a8a9f0a9eab34317b8955c829d188e15
I20260808 07:13:23.480031 21121 raft_consensus.cc:2726] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:13:23.961264 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:23.961019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:58525)
0808 07:13:23.961166 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:23.961199 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260808 07:13:24.017200 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:24.017007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:45081)
0808 07:13:24.017116 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:24.017150 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260808 07:13:24.146443 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:24.146235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:33357)
0808 07:13:24.146373 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:24.146404 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260808 07:13:24.962229 21126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:24.961836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:50633)
0808 07:13:24.962070 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:24.962150 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":129,"threads_started":1}
I20260808 07:13:24.984009 21127 raft_consensus.cc:482] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:24.984143 21127 raft_consensus.cc:504] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:24.984355 21127 leader_election.cc:288] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641), a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921)
W20260808 07:13:24.984874 20524 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
W20260808 07:13:24.985008 20525 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
I20260808 07:13:24.985116 20525 leader_election.cc:302] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eda48abf4ee44127a955bf490181b6d8; no voters: 0d9eeac190ac495c8a5de45a4784db3c, a8a9f0a9eab34317b8955c829d188e15
I20260808 07:13:24.985227 21127 raft_consensus.cc:2726] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:13:25.018108 21129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:25.017751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:57871)
0808 07:13:25.017970 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:25.018050 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":129,"threads_started":1}
I20260808 07:13:25.147552 21131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:25.147207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:49027)
0808 07:13:25.147409 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:25.147464 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":122,"threads_started":1}
I20260808 07:13:25.963001 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:25.962749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:56293)
0808 07:13:25.962918 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:25.962955 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
I20260808 07:13:26.018908 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:26.018643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:54151)
0808 07:13:26.018788 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:26.018858 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260808 07:13:26.148267 21134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:26.148027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:60523)
0808 07:13:26.148182 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:26.148219 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":61,"threads_started":1}
I20260808 07:13:26.963789 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:26.963553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:60737)
0808 07:13:26.963707 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:26.963743 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260808 07:13:27.019543 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:27.019333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:33793)
0808 07:13:27.019467 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:27.019509 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260808 07:13:27.148943 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:27.148728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:45953)
0808 07:13:27.148865 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:27.148898 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260808 07:13:27.964464 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:27.964258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:39425)
0808 07:13:27.964385 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:27.964419 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260808 07:13:28.020124 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:28.019919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:33657)
0808 07:13:28.020052 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:28.020085 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260808 07:13:28.149770 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:28.149540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:53607)
0808 07:13:28.149678 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:28.149723 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260808 07:13:28.965123 21149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:28.964900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:36863)
0808 07:13:28.965042 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:28.965076 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260808 07:13:29.020717 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:29.020520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:56537)
0808 07:13:29.020641 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:29.020676 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260808 07:13:29.150483 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:29.150254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:33027)
0808 07:13:29.150394 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:29.150441 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260808 07:13:29.921226 21155 raft_consensus.cc:482] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:29.921314 21155 raft_consensus.cc:504] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:29.921481 21155 leader_election.cc:288] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641), a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921)
W20260808 07:13:29.921919 20524 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
W20260808 07:13:29.922003 20525 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
I20260808 07:13:29.922066 20525 leader_election.cc:302] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eda48abf4ee44127a955bf490181b6d8; no voters: 0d9eeac190ac495c8a5de45a4784db3c, a8a9f0a9eab34317b8955c829d188e15
I20260808 07:13:29.922155 21155 raft_consensus.cc:2726] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:13:29.965902 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:29.965637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:42775)
0808 07:13:29.965814 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:29.965854 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":90,"threads_started":1}
I20260808 07:13:30.021404 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:30.021142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:58063)
0808 07:13:30.021311 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:30.021357 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260808 07:13:30.151196 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:30.150972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:37291)
0808 07:13:30.151104 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:30.151140 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260808 07:13:30.966670 21160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:30.966434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:34529)
0808 07:13:30.966593 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:30.966624 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260808 07:13:31.022063 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:31.021855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:58875)
0808 07:13:31.021993 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:31.022025 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260808 07:13:31.151849 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:31.151634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:57201)
0808 07:13:31.151766 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:31.151804 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260808 07:13:31.967409 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:31.967173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:43511)
0808 07:13:31.967307 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:31.967345 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260808 07:13:32.022868 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:32.022558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:45631)
0808 07:13:32.022741 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:32.022775 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":117,"threads_started":1}
I20260808 07:13:32.152519 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:32.152306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:43269)
0808 07:13:32.152441 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:32.152476 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260808 07:13:32.968210 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:32.967972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:37387)
0808 07:13:32.968126 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:32.968164 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":66,"threads_started":1}
I20260808 07:13:33.023520 21170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:33.023293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:48183)
0808 07:13:33.023435 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:33.023472 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":61,"threads_started":1}
I20260808 07:13:33.153225 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:33.152959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:34023)
0808 07:13:33.153144 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:33.153181 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":83,"threads_started":1}
I20260808 07:13:33.549074 21172 raft_consensus.cc:482] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:33.549207 21172 raft_consensus.cc:504] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d9eeac190ac495c8a5de45a4784db3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43641 } } peers { permanent_uuid: "a8a9f0a9eab34317b8955c829d188e15" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 35921 } } peers { permanent_uuid: "eda48abf4ee44127a955bf490181b6d8" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 32877 } }
I20260808 07:13:33.549422 21172 leader_election.cc:288] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641), a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921)
W20260808 07:13:33.549880 20524 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d9eeac190ac495c8a5de45a4784db3c (127.30.54.66:43641): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
W20260808 07:13:33.549981 20525 leader_election.cc:341] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8a9f0a9eab34317b8955c829d188e15 (127.30.54.67:35921): Not found: Tablet not found: 4fa077480c45494bb6af9248a0690d2a
I20260808 07:13:33.550042 20525 leader_election.cc:302] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eda48abf4ee44127a955bf490181b6d8; no voters: 0d9eeac190ac495c8a5de45a4784db3c, a8a9f0a9eab34317b8955c829d188e15
I20260808 07:13:33.550109 21172 raft_consensus.cc:2726] T 4fa077480c45494bb6af9248a0690d2a P eda48abf4ee44127a955bf490181b6d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:13:33.969132 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:33.968904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:58029)
0808 07:13:33.969051 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:33.969085 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260808 07:13:34.024206 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:34.024002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:41549)
0808 07:13:34.024127 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:34.024164 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260808 07:13:34.153939 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:34.153738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:59877)
0808 07:13:34.153858 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:34.153895 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260808 07:13:34.969866 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:34.969634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.65:53907)
0808 07:13:34.969789 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:34.969823 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260808 07:13:35.025012 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:35.024738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.66:55939)
0808 07:13:35.024937 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:35.024972 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":62,"threads_started":1}
I20260808 07:13:35.154846 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:35.154600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43295 (local address 127.30.54.67:49561)
0808 07:13:35.154744 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:35.154784 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260808 07:13:35.155063 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 20454
I20260808 07:13:35.166028 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 20691
I20260808 07:13:35.173460 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 20896
2026-08-08T07:13:35Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18641 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:13:35 dist-test-slave-znh6 krb5kdc[21182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:13:35 dist-test-slave-znh6 krb5kdc[21182](info): set up 2 sockets
Aug 08 07:13:35 dist-test-slave-znh6 krb5kdc[21182](info): commencing operation
krb5kdc: starting...
W20260808 07:13:36.416970 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:13:36 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:13:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:13:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:13:36.537678 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45057
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:45057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:36.617852 21200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:36.618083 21200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:36.618124 21200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:36.619357 21200 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:13:36.619415 21200 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:13:36.619436 21200 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:13:36.619464 21200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:36.619498 21200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:36.619527 21200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:36.619560 21200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:36.619587 21200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:36.621033 21200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:45057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:36.621299 21200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:36.621527 21200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:36.626905 21215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:36.626924 21210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:36.626937 21200 server_base.cc:1034] running on GCE node
W20260808 07:13:36.626978 21209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:36.627409 21200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:36.627723 21200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:36.628882 21200 hybrid_clock.cc:648] HybridClock initialized: now 1786173216628862 us; error 44 us; skew 500 ppm
Aug 08 07:13:36 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173216, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:36.631337 21200 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:13:36.631814 21200 webserver.cc:460] Webserver started at http://127.30.54.126:35697/ using document root <none> and password file <none>
I20260808 07:13:36.632041 21200 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:36.632112 21200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:36.632270 21200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:36.633286 21200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "f95fefbd014d42848005fef40e68c7a1"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "2dda25621c6ffec6a99d560b5a38fb9c"
server_key_iv: "5ccb0156ba3de9917e046a811d717687"
server_key_version: "encryptionkey@0"
I20260808 07:13:36.633663 21200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "f95fefbd014d42848005fef40e68c7a1"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "2dda25621c6ffec6a99d560b5a38fb9c"
server_key_iv: "5ccb0156ba3de9917e046a811d717687"
server_key_version: "encryptionkey@0"
I20260808 07:13:36.635100 21200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:13:36.635852 21225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:36.636036 21200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:36.636142 21200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "f95fefbd014d42848005fef40e68c7a1"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "2dda25621c6ffec6a99d560b5a38fb9c"
server_key_iv: "5ccb0156ba3de9917e046a811d717687"
server_key_version: "encryptionkey@0"
I20260808 07:13:36.636215 21200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:36.647871 21200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:36.649881 21200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:36.650054 21200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:36.654372 21200 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:45057
I20260808 07:13:36.654405 21303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:45057 every 8 connection(s)
I20260808 07:13:36.654748 21200 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:36.655717 21304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:36.657963 21304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Bootstrap starting.
I20260808 07:13:36.658579 21304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:36.658906 21304 log.cc:824] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:36.659584 21304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: No bootstrap required, opened a new log
I20260808 07:13:36.661151 21304 raft_consensus.cc:348] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:13:36.661316 21304 raft_consensus.cc:374] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:36.661341 21304 raft_consensus.cc:729] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95fefbd014d42848005fef40e68c7a1, State: Initialized, Role: FOLLOWER
I20260808 07:13:36.661499 21304 consensus_queue.cc:260] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:13:36.661568 21304 raft_consensus.cc:388] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:36.661590 21304 raft_consensus.cc:482] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:36.661641 21304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:36.662052 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 21200
I20260808 07:13:36.662153 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:13:36.662206 21304 raft_consensus.cc:504] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:13:36.662355 21304 leader_election.cc:302] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f95fefbd014d42848005fef40e68c7a1; no voters:
I20260808 07:13:36.662454 30937 external_mini_cluster.cc:1428] Setting key 07f00f483645d4ec83b77c217012d1b6
I20260808 07:13:36.662529 21304 leader_election.cc:288] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:13:36.662592 21309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:36.662757 21309 raft_consensus.cc:686] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 1 LEADER]: Becoming Leader. State: Replica: f95fefbd014d42848005fef40e68c7a1, State: Running, Role: LEADER
I20260808 07:13:36.662781 21304 sys_catalog.cc:564] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:36.662943 21309 consensus_queue.cc:237] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:13:36.663640 21309 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f95fefbd014d42848005fef40e68c7a1. Latest consensus state: current_term: 1 leader_uuid: "f95fefbd014d42848005fef40e68c7a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } } }
I20260808 07:13:36.663648 21310 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f95fefbd014d42848005fef40e68c7a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } } }
I20260808 07:13:36.663827 21310 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:36.664068 21309 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: This master's current role is: LEADER
W20260808 07:13:36.665057 21332 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:13:36.665457 21332 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:13:36.665520 21323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:36.665706 21323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:36.667294 21323 catalog_manager.cc:1347] Generated new cluster ID: 66c7805905a44ca2ba34b6f634bdd77f
I20260808 07:13:36.667346 21323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:13:36 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:36.674201 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:36.663071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.30.54.126:45057)
0808 07:13:36.663282 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:36.663290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:36.663324 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:13:36.663439 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:36.663445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:36.663500 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:36.664482 (+ 982us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:36.664603 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:36.666031 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:36.667016 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:36.667617 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:36.667731 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:36.670333 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:36.670337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:36.670337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:36.670349 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:36.672448 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:36.672846 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:36.672849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:36.672849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:36.672965 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:36.673174 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:36.673175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:36.673175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:36.673476 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:36.673633 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:36.673732 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:13:36.673988 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260808 07:13:36.689070 21323 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:13:36.689780 21323 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:36.702849 21323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Generated new TSK 0
I20260808 07:13:36.703024 21323 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:13:36.741242 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:45057
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:36.822551 21350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:36.822849 21350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:36.822914 21350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:36.824141 21350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:36.824182 21350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:36.824208 21350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:13:36.825691 21350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:36.825930 21350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:36.826148 21350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:36.831058 21355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:36.831071 21358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:36.831101 21350 server_base.cc:1034] running on GCE node
W20260808 07:13:36.831082 21360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:36.831524 21350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:36.831738 21350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:36.832886 21350 hybrid_clock.cc:648] HybridClock initialized: now 1786173216832876 us; error 28 us; skew 500 ppm
Aug 08 07:13:36 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173216, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:36.835297 21350 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:13:36.835749 21350 webserver.cc:460] Webserver started at http://127.30.54.65:33819/ using document root <none> and password file <none>
I20260808 07:13:36.835988 21350 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:36.836057 21350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:36.836213 21350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:36.837174 21350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "539f23a1aba322f66263296ee3bf9f26"
server_key_iv: "e7d45c20134d20a54d2aa4bf830897cd"
server_key_version: "encryptionkey@0"
I20260808 07:13:36.837531 21350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "539f23a1aba322f66263296ee3bf9f26"
server_key_iv: "e7d45c20134d20a54d2aa4bf830897cd"
server_key_version: "encryptionkey@0"
I20260808 07:13:36.838965 21350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:13:36.839802 21369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:36.839982 21350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:36.840044 21350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "539f23a1aba322f66263296ee3bf9f26"
server_key_iv: "e7d45c20134d20a54d2aa4bf830897cd"
server_key_version: "encryptionkey@0"
I20260808 07:13:36.840093 21350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:36.860598 21350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:36.862314 21350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:36.862418 21350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:36.862640 21350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:36.863018 21350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:36.863081 21350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:36.863121 21350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:36.863147 21350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:36.869133 21350 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:45561
I20260808 07:13:36.869208 21525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:45561 every 8 connection(s)
I20260808 07:13:36.869531 21350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:13:36 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173216, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:36.876156 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 21350
I20260808 07:13:36.876247 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:13:36.876415 30937 external_mini_cluster.cc:1428] Setting key 79b5098b818908dc48490344c995b50c
I20260808 07:13:36.878975 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:36.870294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:55483 (local address 127.30.54.126:45057)
0808 07:13:36.870364 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:36.870366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:36.870490 (+ 124us) server_negotiation.cc:407] Connection header received
0808 07:13:36.870961 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:36.870963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:36.870975 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:36.871030 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:36.871900 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:36.872407 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:36.873323 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:36.873524 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:36.873576 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:36.875557 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:36.875559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:36.875559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:36.875568 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:36.877595 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:36.877986 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:36.877988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:36.877988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:36.878049 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:36.878301 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:36.878302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:36.878302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:36.878416 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:36.878494 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:36.878760 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 07:13:36.878876 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:13:36.879104 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:36.870209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:55483)
0808 07:13:36.870370 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:36.870409 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:36.870875 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:36.871078 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:36.871081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:36.871617 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:36.871791 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:36.871794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:36.872497 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:36.872498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:36.873249 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:36.873251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:36.873597 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:36.873598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:36.873692 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:36.874080 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:36.874087 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:36.875457 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:36.877688 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:36.877695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:36.877697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:36.877909 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:36.878119 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:36.878120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:36.878120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:36.878223 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:36.878486 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:36.878493 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:36.878622 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 07:13:36.878723 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:13:36.878891 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260808 07:13:36.879534 21528 heartbeater.cc:347] Connected to a master server at 127.30.54.126:45057
I20260808 07:13:36.879626 21528 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:36.879786 21528 heartbeater.cc:511] Master 127.30.54.126:45057 requested a full tablet report, sending...
I20260808 07:13:36.880290 21248 ts_manager.cc:194] Registered new tserver with Master: c95eb3b1d2984a2d8ea11efcc9121a5a (127.30.54.65:45561)
I20260808 07:13:36.880863 21248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:55483
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:13:36.901496 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:45057
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:36.981042 21535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:36.981232 21535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:36.981256 21535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:36.982300 21535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:36.982338 21535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:36.982365 21535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:13:36.983816 21535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:36.984036 21535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:36.984269 21535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:36.989327 21541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:36.989382 21540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:36.989339 21544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:36.989372 21535 server_base.cc:1034] running on GCE node
I20260808 07:13:36.989717 21535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:36.989986 21535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:36.991154 21535 hybrid_clock.cc:648] HybridClock initialized: now 1786173216991140 us; error 43 us; skew 500 ppm
Aug 08 07:13:36 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173216, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:36.993620 21535 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:13:36.994088 21535 webserver.cc:460] Webserver started at http://127.30.54.66:38549/ using document root <none> and password file <none>
I20260808 07:13:36.994259 21535 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:36.994298 21535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:36.994382 21535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:36.995368 21535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "8e2c5c39164d423b90acd9279feafce4"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "122f25eea7d38f35e619b916defc5527"
server_key_iv: "5a681800ceccc4e9a91e601dabe39579"
server_key_version: "encryptionkey@0"
I20260808 07:13:36.995687 21535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "8e2c5c39164d423b90acd9279feafce4"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "122f25eea7d38f35e619b916defc5527"
server_key_iv: "5a681800ceccc4e9a91e601dabe39579"
server_key_version: "encryptionkey@0"
I20260808 07:13:36.996903 21535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:13:36.997556 21555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:36.997745 21535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:36.997831 21535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "8e2c5c39164d423b90acd9279feafce4"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "122f25eea7d38f35e619b916defc5527"
server_key_iv: "5a681800ceccc4e9a91e601dabe39579"
server_key_version: "encryptionkey@0"
I20260808 07:13:36.997882 21535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:37.034610 21535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:37.036538 21535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:37.036643 21535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:37.036851 21535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:37.037154 21535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:37.037189 21535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:37.037231 21535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:37.037258 21535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:37.043443 21535 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:43605
I20260808 07:13:37.043526 21730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:43605 every 8 connection(s)
I20260808 07:13:37.043826 21535 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:13:37.046304 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 21535
I20260808 07:13:37.046416 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:13:37.046615 30937 external_mini_cluster.cc:1428] Setting key 38050fc48df9a51fcc33933cf4d67f0d
Aug 08 07:13:37 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173216, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:37.053828 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.044526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:37801 (local address 127.30.54.126:45057)
0808 07:13:37.044589 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:37.044591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:37.044804 (+ 213us) server_negotiation.cc:407] Connection header received
0808 07:13:37.045251 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:37.045252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:37.045262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:37.045318 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:37.046209 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.047054 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.048148 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.048459 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.048543 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.051027 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:37.051029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:37.051029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:37.051036 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:37.052491 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:37.052883 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:37.052885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:37.052885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:37.052947 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:37.053199 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:37.053200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:37.053201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:37.053310 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:37.053387 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:37.053650 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 07:13:37.053737 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:13:37.054024 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.044508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:37801)
0808 07:13:37.044671 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:37.044710 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:37.045170 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:37.045368 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:37.045371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:37.045909 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:37.046111 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.046115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.047152 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.047153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.048051 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.048054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.048562 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.048563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.048664 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.049247 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:37.049259 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:37.050923 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:37.052576 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:37.052582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:37.052585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:37.052805 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:37.053024 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:37.053025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:37.053025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:37.053127 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:37.053380 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:37.053386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:37.053522 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 07:13:37.053627 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:13:37.053776 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260808 07:13:37.054646 21731 heartbeater.cc:347] Connected to a master server at 127.30.54.126:45057
I20260808 07:13:37.054724 21731 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:37.054936 21731 heartbeater.cc:511] Master 127.30.54.126:45057 requested a full tablet report, sending...
I20260808 07:13:37.055330 21248 ts_manager.cc:194] Registered new tserver with Master: 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605)
I20260808 07:13:37.055871 21248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:37801
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:13:37.074889 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:45057
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:37.156304 21738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:37.156500 21738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:37.156522 21738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:37.157642 21738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:37.157682 21738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:37.157732 21738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:13:37.159284 21738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:37.159528 21738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:37.159762 21738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:37.164791 21745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:37.164913 21746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:37.164821 21748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:37.165169 21738 server_base.cc:1034] running on GCE node
I20260808 07:13:37.165382 21738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:37.165617 21738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:37.166750 21738 hybrid_clock.cc:648] HybridClock initialized: now 1786173217166743 us; error 30 us; skew 500 ppm
Aug 08 07:13:37 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173217, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:37.169260 21738 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:13:37.169668 21738 webserver.cc:460] Webserver started at http://127.30.54.67:45279/ using document root <none> and password file <none>
I20260808 07:13:37.169885 21738 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:37.169951 21738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:37.170109 21738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:13:37.171165 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "5c7cef30da314ee6b15965e49ccc8b42"
format_stamp: "Formatted at 2026-08-08 07:13:37 on dist-test-slave-znh6"
server_key: "a2e052000f111ba9830f9dae30f35864"
server_key_iv: "bfa326ce964697888bea5da63251a2a7"
server_key_version: "encryptionkey@0"
I20260808 07:13:37.171545 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "5c7cef30da314ee6b15965e49ccc8b42"
format_stamp: "Formatted at 2026-08-08 07:13:37 on dist-test-slave-znh6"
server_key: "a2e052000f111ba9830f9dae30f35864"
server_key_iv: "bfa326ce964697888bea5da63251a2a7"
server_key_version: "encryptionkey@0"
I20260808 07:13:37.172804 21738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:13:37.173508 21761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:37.173674 21738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:37.173775 21738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "5c7cef30da314ee6b15965e49ccc8b42"
format_stamp: "Formatted at 2026-08-08 07:13:37 on dist-test-slave-znh6"
server_key: "a2e052000f111ba9830f9dae30f35864"
server_key_iv: "bfa326ce964697888bea5da63251a2a7"
server_key_version: "encryptionkey@0"
I20260808 07:13:37.173848 21738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:37.196688 21738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:37.198601 21738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:37.198760 21738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:37.199007 21738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:37.199318 21738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:37.199373 21738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:37.199414 21738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:37.199443 21738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:37.205435 21738 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:46275
I20260808 07:13:37.205506 21925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:46275 every 8 connection(s)
I20260808 07:13:37.205794 21738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:13:37.209756 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 21738
I20260808 07:13:37.209849 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:13:37.210018 30937 external_mini_cluster.cc:1428] Setting key 88ca782a253b3183a925b7841ad9724e
Aug 08 07:13:37 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173217, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:37.214846 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.206485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:34183 (local address 127.30.54.126:45057)
0808 07:13:37.206563 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:37.206566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:37.206739 (+ 173us) server_negotiation.cc:407] Connection header received
0808 07:13:37.207281 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:37.207283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:37.207296 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:37.207359 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:37.208262 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.208764 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.209769 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.210038 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.210116 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.212268 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:37.212270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:37.212270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:37.212277 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:37.213448 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:37.213849 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:37.213850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:37.213850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:37.213926 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:37.214183 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:37.214184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:37.214185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:37.214301 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:37.214371 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:37.214649 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 07:13:37.214728 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:13:37.214988 21928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.206438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:34183)
0808 07:13:37.206616 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:37.206660 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:37.207174 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:37.207431 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:37.207435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:37.207983 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:37.208162 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.208166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.208864 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.208865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.209675 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.209677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.210122 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.210124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.210247 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.210778 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:37.210785 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:37.212174 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:37.213533 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:37.213540 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:37.213542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:37.213759 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:37.214003 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:37.214004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:37.214004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:37.214108 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:37.214381 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:37.214388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:37.214513 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 07:13:37.214613 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 07:13:37.214759 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260808 07:13:37.215441 21926 heartbeater.cc:347] Connected to a master server at 127.30.54.126:45057
I20260808 07:13:37.215515 21926 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:37.215684 21926 heartbeater.cc:511] Master 127.30.54.126:45057 requested a full tablet report, sending...
I20260808 07:13:37.216032 21248 ts_manager.cc:194] Registered new tserver with Master: 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275)
I20260808 07:13:37.216341 21248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:34183
I20260808 07:13:37.221601 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:37.228571 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.222870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.30.54.126:45057)
0808 07:13:37.222925 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:37.222926 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:37.223033 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:13:37.223092 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:37.223093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:37.223105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:37.223175 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:37.223685 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.224209 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.224911 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.225097 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.225154 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.226245 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:37.226246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:37.226246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:37.226252 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:37.227422 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:37.227759 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:37.227760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:37.227760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:37.227821 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:37.228018 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:37.228019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:37.228020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:37.228160 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:37.228272 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:37.228397 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 07:13:37.228473 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:13:37.230229 21248 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:13:37.231868 21248 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:13:37.238144 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.234354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:43605 (local address 127.0.0.1:54406)
0808 07:13:37.234710 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:37.234721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:37.234764 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:37.235047 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:37.235048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:37.235050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:37.235085 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.235087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.236294 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.236295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.237163 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.237166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.237819 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.237820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.237900 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.237913 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:13:37.238018 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:13:37.238073 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":83,"threads_started":1}
I20260808 07:13:37.238153 21947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.234587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54406 (local address 127.30.54.66:43605)
0808 07:13:37.234781 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:37.234786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:37.234798 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:13:37.234857 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:37.234860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:37.234876 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:37.234954 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:37.235162 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.236190 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.237231 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.237714 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.237814 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.237874 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:37.238038 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 07:13:37.238078 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260808 07:13:37.238340 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.234169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:45561 (local address 127.0.0.1:48436)
0808 07:13:37.234477 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:37.234490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:37.234543 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:37.235112 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:37.235113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:37.235116 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:37.235178 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.235180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.236071 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.236072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.237158 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.237160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.238089 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.238091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.238150 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.238155 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:13:37.238184 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 07:13:37.238222 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":26,"thread_start_us":79,"threads_started":1}
I20260808 07:13:37.238654 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.234257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.30.54.65:45561)
0808 07:13:37.234930 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:37.234935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:37.234945 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:37.234987 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:37.234989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:37.235004 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:37.235262 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:37.235326 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.235961 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.237236 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.237989 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.238414 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.238497 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:37.238527 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 07:13:37.238575 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":93,"threads_started":1}
I20260808 07:13:37.239156 21630 tablet_service.cc:1467] Processing CreateTablet for tablet e6f0402f071146d391d3e6c23082ebf4 (DEFAULT_TABLE table=test-table [id=405fd75cae1641fda769acc529c1056b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:37.239275 21433 tablet_service.cc:1467] Processing CreateTablet for tablet e6f0402f071146d391d3e6c23082ebf4 (DEFAULT_TABLE table=test-table [id=405fd75cae1641fda769acc529c1056b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:37.239547 21630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f0402f071146d391d3e6c23082ebf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:37.239624 21433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f0402f071146d391d3e6c23082ebf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:37.239811 21940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.234152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:46275 (local address 127.0.0.1:48586)
0808 07:13:37.234311 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:37.234355 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:37.234422 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:37.234617 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:37.234619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:37.234621 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:37.234700 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.234702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.236026 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.236027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.238216 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.238218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.239575 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.239576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.239650 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.239653 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:13:37.239709 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:13:37.239748 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":1032,"thread_start_us":60,"threads_started":1}
I20260808 07:13:37.240059 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.234287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48586 (local address 127.30.54.67:46275)
0808 07:13:37.234464 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:37.234469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:37.234482 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:13:37.234517 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:37.234519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:37.234533 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:37.234611 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:37.234795 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.235910 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.238667 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.239404 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.239484 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.239536 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:37.239934 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 07:13:37.239979 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
I20260808 07:13:37.240708 21830 tablet_service.cc:1467] Processing CreateTablet for tablet e6f0402f071146d391d3e6c23082ebf4 (DEFAULT_TABLE table=test-table [id=405fd75cae1641fda769acc529c1056b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:37.241004 21830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f0402f071146d391d3e6c23082ebf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:37.242919 21948 tablet_bootstrap.cc:492] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Bootstrap starting.
I20260808 07:13:37.242919 21950 tablet_bootstrap.cc:492] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: Bootstrap starting.
I20260808 07:13:37.243671 21948 tablet_bootstrap.cc:654] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:37.243671 21950 tablet_bootstrap.cc:654] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:37.244100 21951 tablet_bootstrap.cc:492] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: Bootstrap starting.
I20260808 07:13:37.244300 21948 log.cc:824] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:37.244297 21950 log.cc:824] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:37.244872 21951 tablet_bootstrap.cc:654] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:37.245136 21948 tablet_bootstrap.cc:492] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: No bootstrap required, opened a new log
I20260808 07:13:37.245194 21948 ts_tablet_manager.cc:1397] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:13:37.245481 21951 log.cc:824] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:37.245877 21950 tablet_bootstrap.cc:492] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: No bootstrap required, opened a new log
I20260808 07:13:37.245949 21950 ts_tablet_manager.cc:1397] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:13:37.247329 21951 tablet_bootstrap.cc:492] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: No bootstrap required, opened a new log
I20260808 07:13:37.247242 21948 raft_consensus.cc:348] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:37.247411 21951 ts_tablet_manager.cc:1397] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:13:37.247429 21948 raft_consensus.cc:374] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:37.247465 21948 raft_consensus.cc:729] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95eb3b1d2984a2d8ea11efcc9121a5a, State: Initialized, Role: FOLLOWER
I20260808 07:13:37.247570 21948 consensus_queue.cc:260] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:37.247866 21528 heartbeater.cc:503] Master 127.30.54.126:45057 was elected leader, sending a full tablet report...
I20260808 07:13:37.247871 21948 ts_tablet_manager.cc:1428] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:13:37.247922 21950 raft_consensus.cc:348] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:37.248087 21950 raft_consensus.cc:374] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:37.248132 21950 raft_consensus.cc:729] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e2c5c39164d423b90acd9279feafce4, State: Initialized, Role: FOLLOWER
I20260808 07:13:37.248297 21950 consensus_queue.cc:260] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:37.248584 21731 heartbeater.cc:503] Master 127.30.54.126:45057 was elected leader, sending a full tablet report...
I20260808 07:13:37.248582 21950 ts_tablet_manager.cc:1428] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:13:37.249337 21951 raft_consensus.cc:348] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:37.249459 21951 raft_consensus.cc:374] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:37.249509 21951 raft_consensus.cc:729] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7cef30da314ee6b15965e49ccc8b42, State: Initialized, Role: FOLLOWER
I20260808 07:13:37.249615 21951 consensus_queue.cc:260] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:37.249882 21951 ts_tablet_manager.cc:1428] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:13:37.249912 21926 heartbeater.cc:503] Master 127.30.54.126:45057 was elected leader, sending a full tablet report...
W20260808 07:13:37.294423 21732 tablet.cc:2369] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:13:37.370299 21529 tablet.cc:2369] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:13:37.456363 21927 tablet.cc:2369] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:13:37.468333 21960 raft_consensus.cc:482] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:37.468492 21960 raft_consensus.cc:504] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:37.468868 21960 leader_election.cc:288] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275), 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605)
I20260808 07:13:37.473196 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.469161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:58153 (local address 127.30.54.67:46275)
0808 07:13:37.469253 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:37.469255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:37.469487 (+ 232us) server_negotiation.cc:407] Connection header received
0808 07:13:37.469521 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:37.469522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:37.469532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:37.469595 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:37.469815 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.470582 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.472213 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.472784 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.472864 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.472873 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:37.473088 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 07:13:37.473126 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 07:13:37.473196 21947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.469413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:42861 (local address 127.30.54.66:43605)
0808 07:13:37.469479 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:37.469481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:37.469492 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:13:37.469521 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:37.469522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:37.469532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:37.469595 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:37.469814 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.470574 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.472213 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.472784 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.472864 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.472873 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:37.473088 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 07:13:37.473125 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:13:37.473260 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.469094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:46275 (local address 127.30.54.65:58153)
0808 07:13:37.469249 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:37.469261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:37.469378 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:37.469621 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:37.469622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:37.469625 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:37.469740 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.469742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.470678 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.470679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.472118 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.472121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.472891 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.472893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.472993 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.473000 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:13:37.473066 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:13:37.473135 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":126,"thread_start_us":77,"threads_started":1}
I20260808 07:13:37.473469 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.469297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:43605 (local address 127.30.54.65:42861)
0808 07:13:37.469421 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:37.469431 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:37.469483 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:37.469637 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:37.469637 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:37.469638 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:37.469734 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.469736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.470678 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.470679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.472118 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:37.472121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.472893 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.472894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:37.472993 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.473002 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:13:37.473066 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:13:37.473134 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":69,"thread_start_us":81,"threads_started":1}
I20260808 07:13:37.473690 21860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f0402f071146d391d3e6c23082ebf4" candidate_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7cef30da314ee6b15965e49ccc8b42" is_pre_election: true
I20260808 07:13:37.473865 21860 raft_consensus.cc:2454] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95eb3b1d2984a2d8ea11efcc9121a5a in term 0.
I20260808 07:13:37.474174 21370 leader_election.cc:302] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7cef30da314ee6b15965e49ccc8b42, c95eb3b1d2984a2d8ea11efcc9121a5a; no voters:
I20260808 07:13:37.474349 21960 raft_consensus.cc:2781] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:37.474308 21655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f0402f071146d391d3e6c23082ebf4" candidate_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e2c5c39164d423b90acd9279feafce4" is_pre_election: true
I20260808 07:13:37.474421 21960 raft_consensus.cc:482] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:37.474452 21960 raft_consensus.cc:3037] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:37.474476 21655 raft_consensus.cc:2454] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95eb3b1d2984a2d8ea11efcc9121a5a in term 0.
I20260808 07:13:37.475243 21960 raft_consensus.cc:504] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:37.475390 21960 leader_election.cc:288] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 1 election: Requested vote from peers 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275), 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605)
I20260808 07:13:37.475579 21655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f0402f071146d391d3e6c23082ebf4" candidate_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e2c5c39164d423b90acd9279feafce4"
I20260808 07:13:37.475593 21860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f0402f071146d391d3e6c23082ebf4" candidate_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7cef30da314ee6b15965e49ccc8b42"
I20260808 07:13:37.475661 21655 raft_consensus.cc:3037] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:37.475684 21860 raft_consensus.cc:3037] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:37.476373 21655 raft_consensus.cc:2454] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95eb3b1d2984a2d8ea11efcc9121a5a in term 1.
I20260808 07:13:37.476470 21860 raft_consensus.cc:2454] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95eb3b1d2984a2d8ea11efcc9121a5a in term 1.
I20260808 07:13:37.476533 21370 leader_election.cc:302] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e2c5c39164d423b90acd9279feafce4, c95eb3b1d2984a2d8ea11efcc9121a5a; no voters:
I20260808 07:13:37.476624 21960 raft_consensus.cc:2781] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:37.476809 21960 raft_consensus.cc:686] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 LEADER]: Becoming Leader. State: Replica: c95eb3b1d2984a2d8ea11efcc9121a5a, State: Running, Role: LEADER
I20260808 07:13:37.476908 21960 consensus_queue.cc:237] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:37.477816 21247 catalog_manager.cc:5495] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a reported cstate change: term changed from 0 to 1, leader changed from <none> to c95eb3b1d2984a2d8ea11efcc9121a5a (127.30.54.65). New cstate: current_term: 1 leader_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } health_report { overall_health: UNKNOWN } } }
I20260808 07:13:37.519519 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:37.517333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.30.54.65:45561)
0808 07:13:37.517400 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:37.517401 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:37.517410 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:37.517450 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:37.517451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:37.517469 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:37.517531 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:37.517643 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.518103 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.518799 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:37.519001 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:37.519053 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:37.519195 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:37.519298 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:37.519360 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:37.519397 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:13:37.519413 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:13:37.521447 21655 raft_consensus.cc:1261] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [term 1 FOLLOWER]: Refusing update from remote peer c95eb3b1d2984a2d8ea11efcc9121a5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:37.521447 21860 raft_consensus.cc:1261] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [term 1 FOLLOWER]: Refusing update from remote peer c95eb3b1d2984a2d8ea11efcc9121a5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:37.521811 21960 consensus_queue.cc:1035] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:37.521977 21972 consensus_queue.cc:1035] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:37.524361 21977 mvcc.cc:205] Tried to move back new op lower bound from 7316165498964848640 to 7316165498786021376. Current Snapshot: MvccSnapshot[applied={T|T < 7316165498964848640}]
I20260808 07:13:37.524925 21978 mvcc.cc:205] Tried to move back new op lower bound from 7316165498964848640 to 7316165498786021376. Current Snapshot: MvccSnapshot[applied={T|T < 7316165498964848640}]
I20260808 07:13:37.526626 21247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:37.526726 21247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:37.527660 21247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Sending DeleteTablet for 3 replicas of tablet e6f0402f071146d391d3e6c23082ebf4
I20260808 07:13:37.528021 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 21350
I20260808 07:13:37.528010 21830 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f0402f071146d391d3e6c23082ebf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:37 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48586
I20260808 07:13:37.528079 21433 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f0402f071146d391d3e6c23082ebf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:37 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:48436
I20260808 07:13:37.530331 21630 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f0402f071146d391d3e6c23082ebf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:37 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:54406
I20260808 07:13:37.536625 21992 tablet_replica.cc:330] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: stopping tablet replica
I20260808 07:13:37.536657 21991 tablet_replica.cc:330] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: stopping tablet replica
I20260808 07:13:37.536762 21991 raft_consensus.cc:2229] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:37.536758 21992 raft_consensus.cc:2229] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:37.536875 21991 raft_consensus.cc:2258] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:37.536886 21992 raft_consensus.cc:2258] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:37.537289 21991 ts_tablet_manager.cc:1905] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:37.539033 21991 ts_tablet_manager.cc:1918] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:37.539139 21991 log.cc:1196] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/e6f0402f071146d391d3e6c23082ebf4
I20260808 07:13:37.539256 21992 ts_tablet_manager.cc:1905] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:37.539475 21991 ts_tablet_manager.cc:1939] T e6f0402f071146d391d3e6c23082ebf4 P 5c7cef30da314ee6b15965e49ccc8b42: Deleting consensus metadata
I20260808 07:13:37.539959 21226 catalog_manager.cc:4841] TS 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275): tablet e6f0402f071146d391d3e6c23082ebf4 (table test-table [id=405fd75cae1641fda769acc529c1056b]) successfully deleted
I20260808 07:13:37.541312 21992 ts_tablet_manager.cc:1918] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:37.541396 21992 log.cc:1196] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/e6f0402f071146d391d3e6c23082ebf4
I20260808 07:13:37.541661 21992 ts_tablet_manager.cc:1939] T e6f0402f071146d391d3e6c23082ebf4 P 8e2c5c39164d423b90acd9279feafce4: Deleting consensus metadata
W20260808 07:13:37.541782 21227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:45561 (error 108)
I20260808 07:13:37.542106 21226 catalog_manager.cc:4841] TS 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605): tablet e6f0402f071146d391d3e6c23082ebf4 (table test-table [id=405fd75cae1641fda769acc529c1056b]) successfully deleted
W20260808 07:13:37.542367 21940 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:13:37.542182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:45561 (local address 127.0.0.1:48448)
0808 07:13:37.542220 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:37.542231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:37.542276 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:37.542314 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:45561: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 07:13:37.542486 21227 catalog_manager.cc:4584] TS c95eb3b1d2984a2d8ea11efcc9121a5a (127.30.54.65:45561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6f0402f071146d391d3e6c23082ebf4: Network error: Client connection negotiation failed: client connection to 127.30.54.65:45561: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260808 07:13:37.542722 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 21535
I20260808 07:13:37.549288 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 21738
I20260808 07:13:37.571583 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 21200
I20260808 07:13:57.583974 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45057
--webserver_interface=127.30.54.126
--webserver_port=35697
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:45057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:13:57.674506 21994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:57.674721 21994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:57.674746 21994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:57.676038 21994 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:13:57.676083 21994 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:13:57.676098 21994 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:13:57.676156 21994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:13:57.676179 21994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:57.676208 21994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:57.676244 21994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:13:57.676283 21994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:13:57.677938 21994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:45057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=35697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:57.678184 21994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:57.678423 21994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:57.688215 22005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:57.688251 22007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:57.688201 22004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:57.688498 21994 server_base.cc:1034] running on GCE node
I20260808 07:13:57.688814 21994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:57.689178 21994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:57.690331 21994 hybrid_clock.cc:648] HybridClock initialized: now 1786173237690305 us; error 45 us; skew 500 ppm
Aug 08 07:13:57 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173237, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:57.693096 21994 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:13:57.693620 21994 webserver.cc:460] Webserver started at http://127.30.54.126:35697/ using document root <none> and password file <none>
I20260808 07:13:57.693892 21994 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:57.693986 21994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:57.695592 21994 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:57.696292 22015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:57.696491 21994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:13:57.696596 21994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "f95fefbd014d42848005fef40e68c7a1"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "2dda25621c6ffec6a99d560b5a38fb9c"
server_key_iv: "5ccb0156ba3de9917e046a811d717687"
server_key_version: "encryptionkey@0"
I20260808 07:13:57.697005 21994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:57.715301 21994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:57.717140 21994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:57.717307 21994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:57.721530 21994 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:45057
I20260808 07:13:57.721562 22092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:45057 every 8 connection(s)
I20260808 07:13:57.721953 21994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:13:57.723555 22093 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:13:57.724156 22093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Bootstrap starting.
I20260808 07:13:57.730269 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 21994
I20260808 07:13:57.730599 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:45561
--local_ip_for_outbound_sockets=127.30.54.65
--tserver_master_addrs=127.30.54.126:45057
--webserver_port=33819
--webserver_interface=127.30.54.65
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:57.732689 22093 log.cc:824] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:57.734558 22093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:57.734941 22093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Bootstrap complete.
I20260808 07:13:57.737169 22093 raft_consensus.cc:348] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:13:57.737416 22093 raft_consensus.cc:729] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95fefbd014d42848005fef40e68c7a1, State: Initialized, Role: FOLLOWER
I20260808 07:13:57.737602 22093 consensus_queue.cc:260] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:13:57.737704 22093 raft_consensus.cc:388] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:13:57.737766 22093 raft_consensus.cc:482] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:13:57.737818 22093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:13:57.738796 22093 raft_consensus.cc:504] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:13:57.738973 22093 leader_election.cc:302] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f95fefbd014d42848005fef40e68c7a1; no voters:
I20260808 07:13:57.739125 22093 leader_election.cc:288] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:13:57.739161 22098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:13:57.739280 22098 raft_consensus.cc:686] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 2 LEADER]: Becoming Leader. State: Replica: f95fefbd014d42848005fef40e68c7a1, State: Running, Role: LEADER
I20260808 07:13:57.739387 22093 sys_catalog.cc:564] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:13:57.739365 22098 consensus_queue.cc:237] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:13:57.739632 22098 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f95fefbd014d42848005fef40e68c7a1. Latest consensus state: current_term: 2 leader_uuid: "f95fefbd014d42848005fef40e68c7a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } } }
I20260808 07:13:57.739696 22098 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:57.739655 22099 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f95fefbd014d42848005fef40e68c7a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } } }
I20260808 07:13:57.739754 22099 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:13:57.740617 22102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:13:57.741551 22102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:13:57.741676 22102 catalog_manager.cc:1259] Loaded cluster ID: 66c7805905a44ca2ba34b6f634bdd77f
I20260808 07:13:57.741942 22102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:13:57.743220 22102 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:13:57.743419 22102 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Loaded TSK: 0
I20260808 07:13:57.762205 22102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Generated new TSK 1
I20260808 07:13:57.762312 22102 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:13:57.822088 22096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:57.822299 22096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:57.822325 22096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:57.823740 22096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:57.823791 22096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:57.823820 22096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:13:57.825426 22096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:45561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=33819
--tserver_master_addrs=127.30.54.126:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:57.825670 22096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:57.825934 22096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:57.832296 22130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:57.832278 22129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:57.832485 22096 server_base.cc:1034] running on GCE node
W20260808 07:13:57.832639 22133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:57.832829 22096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:57.833105 22096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:57.834296 22096 hybrid_clock.cc:648] HybridClock initialized: now 1786173237834267 us; error 43 us; skew 500 ppm
Aug 08 07:13:57 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173237, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:57.837313 22096 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:13:57.837841 22096 webserver.cc:460] Webserver started at http://127.30.54.65:33819/ using document root <none> and password file <none>
I20260808 07:13:57.838063 22096 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:57.838148 22096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:57.839671 22096 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:13:57.840557 22145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:57.840745 22096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:57.840845 22096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "539f23a1aba322f66263296ee3bf9f26"
server_key_iv: "e7d45c20134d20a54d2aa4bf830897cd"
server_key_version: "encryptionkey@0"
I20260808 07:13:57.841235 22096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:57.860855 22096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:57.863229 22096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:57.863418 22096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:57.863670 22096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:57.864166 22159 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:13:57.865636 22096 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:13:57.865692 22096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:13:57.865718 22096 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:13:57.866470 22096 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:13:57.866516 22096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:57.866561 22159 tablet_bootstrap.cc:492] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Bootstrap starting.
I20260808 07:13:57.874614 22096 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:45561
I20260808 07:13:57.874691 22310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:45561 every 8 connection(s)
I20260808 07:13:57.875138 22096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:13:57.876503 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 22096
I20260808 07:13:57.876822 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:43605
--local_ip_for_outbound_sockets=127.30.54.66
--tserver_master_addrs=127.30.54.126:45057
--webserver_port=38549
--webserver_interface=127.30.54.66
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:13:57.878124 22159 log.cc:824] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:57.880573 22159 tablet_bootstrap.cc:492] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:13:57.880909 22159 tablet_bootstrap.cc:492] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Bootstrap complete.
I20260808 07:13:57.881991 22159 ts_tablet_manager.cc:1397] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.007s
I20260808 07:13:57.883234 22159 raft_consensus.cc:348] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:57.883509 22159 raft_consensus.cc:729] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95eb3b1d2984a2d8ea11efcc9121a5a, State: Initialized, Role: FOLLOWER
Aug 08 07:13:57 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173237, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:57.883651 22159 consensus_queue.cc:260] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } }
I20260808 07:13:57.883913 22159 ts_tablet_manager.cc:1428] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
I20260808 07:13:57.888402 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:57.875837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:34395)
0808 07:13:57.876048 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:57.876098 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:57.876891 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:57.877522 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:57.877527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:57.877918 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:57.878186 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:57.878192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:57.879578 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:57.879579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:57.880858 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:57.880861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:57.881475 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:57.881476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:57.881624 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:57.882167 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:57.882176 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:57.884046 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:57.886286 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:57.886292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:57.886294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:57.886503 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:57.886799 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:57.886800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:57.886800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:57.887153 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:57.887688 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:57.887696 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:57.887880 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 07:13:57.888005 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:13:57.888182 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260808 07:13:57.888612 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:57.876015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:34395 (local address 127.30.54.126:45057)
0808 07:13:57.876183 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:57.876191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:57.876227 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:13:57.877189 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:57.877200 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:57.877268 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:57.877439 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:57.878329 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:57.879450 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:57.880943 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:57.881365 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:57.881463 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:57.884151 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:57.884155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:57.884155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:57.884167 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:57.886193 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:57.886592 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:57.886595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:57.886595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:57.886719 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:57.887223 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:57.887225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:57.887225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:57.887547 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:57.887682 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:57.888097 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 07:13:57.888316 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260808 07:13:57.889211 22311 heartbeater.cc:347] Connected to a master server at 127.30.54.126:45057
I20260808 07:13:57.889294 22311 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:57.889508 22311 heartbeater.cc:511] Master 127.30.54.126:45057 requested a full tablet report, sending...
I20260808 07:13:57.891103 22036 ts_manager.cc:194] Registered new tserver with Master: c95eb3b1d2984a2d8ea11efcc9121a5a (127.30.54.65:45561)
I20260808 07:13:57.892087 22036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:34395
I20260808 07:13:57.893103 22311 heartbeater.cc:503] Master 127.30.54.126:45057 was elected leader, sending a full tablet report...
Aug 08 07:13:57 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173237, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.65@KRBTEST.COM
I20260808 07:13:57.899853 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:57.891630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:45561 (local address 127.0.0.1:50614)
0808 07:13:57.891813 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:57.891861 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:57.891941 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:57.892689 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:57.892695 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:57.892942 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:57.892984 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:57.892985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:57.894014 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:57.894015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:57.894573 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:57.894575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:57.895115 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:57.895116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:57.895200 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:57.895627 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:57.895631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:57.897086 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:57.898691 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:57.898698 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:57.898700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:57.898997 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:57.899203 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:57.899204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:57.899205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:57.899269 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:57.899503 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:57.899509 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:57.899558 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 07:13:57.899640 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:13:57.899750 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260808 07:13:57.900177 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:57.892370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.30.54.65:45561)
0808 07:13:57.892530 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:57.892534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:57.892544 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:57.892579 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:57.892580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:57.892593 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:57.892672 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:57.893072 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:57.893929 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:57.894679 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:57.895035 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:57.895115 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:57.897164 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:57.897171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:57.897171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:57.897177 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:57.898610 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:57.899085 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:57.899087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:57.899087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:57.899138 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:57.899327 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:57.899327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:57.899328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:57.899442 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:57.899501 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:57.899999 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 07:13:57.900090 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260808 07:13:57.900520 22219 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f0402f071146d391d3e6c23082ebf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:37 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:50614
I20260808 07:13:57.901124 22325 tablet_replica.cc:330] stopping tablet replica
I20260808 07:13:57.901235 22325 raft_consensus.cc:2229] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:57.901302 22325 raft_consensus.cc:2258] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:57.901585 22325 ts_tablet_manager.cc:1905] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:57.903825 22325 ts_tablet_manager.cc:1918] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:57.903928 22325 log.cc:1196] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal/wals/e6f0402f071146d391d3e6c23082ebf4
I20260808 07:13:57.904176 22325 ts_tablet_manager.cc:1939] T e6f0402f071146d391d3e6c23082ebf4 P c95eb3b1d2984a2d8ea11efcc9121a5a: Deleting consensus metadata
I20260808 07:13:57.904708 22017 catalog_manager.cc:4841] TS c95eb3b1d2984a2d8ea11efcc9121a5a (127.30.54.65:45561): tablet e6f0402f071146d391d3e6c23082ebf4 (table test-table [id=405fd75cae1641fda769acc529c1056b]) successfully deleted
W20260808 07:13:57.967113 22320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:57.967296 22320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:57.967319 22320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:57.968375 22320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:57.968416 22320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:57.968442 22320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:13:57.969771 22320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:43605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=38549
--tserver_master_addrs=127.30.54.126:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:57.969941 22320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:57.970111 22320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:57.975260 22331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:57.975306 22334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:57.975370 22320 server_base.cc:1034] running on GCE node
W20260808 07:13:57.975244 22332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:57.975656 22320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:57.975915 22320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:57.977095 22320 hybrid_clock.cc:648] HybridClock initialized: now 1786173237977089 us; error 39 us; skew 500 ppm
Aug 08 07:13:57 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173237, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:57.979494 22320 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:13:57.979969 22320 webserver.cc:460] Webserver started at http://127.30.54.66:38549/ using document root <none> and password file <none>
I20260808 07:13:57.980206 22320 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:57.980290 22320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:57.981637 22320 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:57.982261 22349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:57.982427 22320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:57.982527 22320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "8e2c5c39164d423b90acd9279feafce4"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "122f25eea7d38f35e619b916defc5527"
server_key_iv: "5a681800ceccc4e9a91e601dabe39579"
server_key_version: "encryptionkey@0"
I20260808 07:13:57.982913 22320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:57.993656 22320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:57.995684 22320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:57.995859 22320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:57.996093 22320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:57.996418 22320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:57.996476 22320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:57.996517 22320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:57.996544 22320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:58.003063 22320 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:43605
I20260808 07:13:58.003118 22522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:43605 every 8 connection(s)
I20260808 07:13:58.003485 22320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:13:58.007922 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 22320
I20260808 07:13:58.008261 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:46275
--local_ip_for_outbound_sockets=127.30.54.67
--tserver_master_addrs=127.30.54.126:45057
--webserver_port=45279
--webserver_interface=127.30.54.67
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:13:58 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173237, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:58.013372 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.004295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:50163 (local address 127.30.54.126:45057)
0808 07:13:58.004342 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.004344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.004536 (+ 192us) server_negotiation.cc:407] Connection header received
0808 07:13:58.004989 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.004991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.005002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.005051 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:58.005928 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.006445 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.007488 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.007695 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.007750 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.010284 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:58.010286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:58.010286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:58.010293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:58.011807 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:58.012325 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:58.012326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:58.012326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:58.012383 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:58.012697 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:58.012698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:58.012698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:58.012802 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:58.012898 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:58.013184 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 07:13:58.013277 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:13:58.013551 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.004273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:50163)
0808 07:13:58.004417 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.004451 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:58.004903 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:58.005106 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:58.005108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:58.005658 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:58.005846 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.005850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.006531 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.006532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.007416 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.007418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.007795 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.007797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.007952 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.008426 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:58.008437 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:58.010161 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:58.011894 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:58.011902 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:58.011904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:58.012226 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:58.012463 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:58.012464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:58.012465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:58.012620 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:58.012890 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:58.012896 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:58.013041 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 07:13:58.013162 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 07:13:58.013318 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260808 07:13:58.014041 22523 heartbeater.cc:347] Connected to a master server at 127.30.54.126:45057
I20260808 07:13:58.014128 22523 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:58.014304 22523 heartbeater.cc:511] Master 127.30.54.126:45057 requested a full tablet report, sending...
I20260808 07:13:58.014695 22036 ts_manager.cc:194] Registered new tserver with Master: 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605)
I20260808 07:13:58.015096 22036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:50163
W20260808 07:13:58.090704 22526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:13:58.090986 22526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:13:58.091029 22526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:13:58.092144 22526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:13:58.092213 22526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:13:58.092259 22526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:13:58.093685 22526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:46275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=45279
--tserver_master_addrs=127.30.54.126:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:13:58.093951 22526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:13:58.094182 22526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:13:58.099225 22532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:13:58.099246 22531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:58.099284 22526 server_base.cc:1034] running on GCE node
W20260808 07:13:58.099236 22537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:13:58.099722 22526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:13:58.099924 22526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:13:58.101055 22526 hybrid_clock.cc:648] HybridClock initialized: now 1786173238101046 us; error 27 us; skew 500 ppm
Aug 08 07:13:58 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173238, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:13:58.103559 22526 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:13:58.104035 22526 webserver.cc:460] Webserver started at http://127.30.54.67:45279/ using document root <none> and password file <none>
I20260808 07:13:58.104260 22526 fs_manager.cc:359] Metadata directory not provided
I20260808 07:13:58.104350 22526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:13:58.105650 22526 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:13:58.106283 22546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:58.106475 22526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:58.106571 22526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "5c7cef30da314ee6b15965e49ccc8b42"
format_stamp: "Formatted at 2026-08-08 07:13:37 on dist-test-slave-znh6"
server_key: "a2e052000f111ba9830f9dae30f35864"
server_key_iv: "bfa326ce964697888bea5da63251a2a7"
server_key_version: "encryptionkey@0"
I20260808 07:13:58.106945 22526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:13:58.125970 22526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:13:58.128245 22526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:13:58.128412 22526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:13:58.128634 22526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:13:58.128937 22526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:13:58.128993 22526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:58.129033 22526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:13:58.129060 22526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:13:58.135076 22526 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:46275
I20260808 07:13:58.135128 22713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:46275 every 8 connection(s)
I20260808 07:13:58.135502 22526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:13:58 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173238, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:13:58.144757 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 22526
I20260808 07:13:58.145165 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.136208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:42521 (local address 127.30.54.126:45057)
0808 07:13:58.136296 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.136298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.136490 (+ 192us) server_negotiation.cc:407] Connection header received
0808 07:13:58.137016 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.137017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.137030 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.137089 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:58.138018 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.138524 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.139756 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.139977 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.140085 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.142204 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:58.142206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:58.142206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:58.142214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:58.143502 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:58.143961 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:58.143963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:58.143964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:58.144023 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:58.144324 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:58.144325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:58.144325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:58.144449 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:58.144526 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:58.144914 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 07:13:58.145044 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:13:58.145255 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.136181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:42521)
0808 07:13:58.136353 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.136400 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:58.136906 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:58.137151 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:58.137155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:58.137715 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:13:58.137911 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.137915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.138626 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.138627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.139662 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.139665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.140091 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.140093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.140231 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.140704 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:13:58.140713 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:13:58.142106 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:13:58.143624 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:58.143631 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:58.143633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:58.143866 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:58.144094 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:13:58.144095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:13:58.144095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:13:58.144250 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:13:58.144528 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:13:58.144534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:13:58.144719 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 07:13:58.144860 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 07:13:58.145030 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260808 07:13:58.145705 22714 heartbeater.cc:347] Connected to a master server at 127.30.54.126:45057
I20260808 07:13:58.145814 22714 heartbeater.cc:464] Registering TS with master...
I20260808 07:13:58.146030 22714 heartbeater.cc:511] Master 127.30.54.126:45057 requested a full tablet report, sending...
I20260808 07:13:58.146461 22036 ts_manager.cc:194] Registered new tserver with Master: 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275)
I20260808 07:13:58.146791 22036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:42521
I20260808 07:13:58.152076 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.145204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.30.54.126:45057)
0808 07:13:58.145241 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.145244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.145463 (+ 219us) server_negotiation.cc:407] Connection header received
0808 07:13:58.145589 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.145590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.145600 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.145652 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:58.146909 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.147619 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.148396 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.148573 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.148629 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.149674 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:58.149675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:58.149675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:58.149681 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:58.151119 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:58.151438 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:58.151439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:58.151439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:58.151489 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:58.151654 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:58.151655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:58.151655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:58.151744 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:58.151862 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:58.151929 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:13:58.151984 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20,"thread_start_us":60,"threads_started":1}
I20260808 07:13:58.152835 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:13:58.155416 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.153207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48856 (local address 127.30.54.126:45057)
0808 07:13:58.153294 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.153295 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.153418 (+ 123us) server_negotiation.cc:407] Connection header received
0808 07:13:58.153452 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.153453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.153482 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.153537 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:58.153627 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.154044 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.154743 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.154988 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.155040 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.155179 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:58.155213 (+ 34us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 07:13:58.155319 (+ 106us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48856: token expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 07:13:58.155478 22317 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48856: token expired
W20260808 07:13:58.155616 21932 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.54.126:45057)
I20260808 07:13:58.161067 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.155853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.30.54.126:45057)
0808 07:13:58.155886 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.155888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.155894 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:13:58.155947 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.155947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.155953 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.155994 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:13:58.156405 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.156812 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.157488 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.157682 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.157746 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.158931 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:13:58.158932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:13:58.158932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:13:58.158939 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:13:58.160101 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:58.160426 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:58.160427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:58.160427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:58.160473 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:13:58.160708 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:13:58.160708 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:13:58.160709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:13:58.160800 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:13:58.160885 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:58.160962 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:13:58.161014 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:13:58.162293 22036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:13:58.163997 22036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:13:58.166149 22219 tablet_service.cc:1467] Processing CreateTablet for tablet 9130591296d94840bbe01d2c4bdc2103 (DEFAULT_TABLE table=test-table [id=a0ee20f0ec7c421c9d2c77314fa411ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:58.166381 22219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9130591296d94840bbe01d2c4bdc2103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:58.167970 22159 tablet_bootstrap.cc:492] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Bootstrap starting.
I20260808 07:13:58.168301 22159 tablet_bootstrap.cc:654] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:58.168825 22159 tablet_bootstrap.cc:492] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: No bootstrap required, opened a new log
I20260808 07:13:58.168885 22159 ts_tablet_manager.cc:1397] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:58.169209 22159 raft_consensus.cc:348] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:13:58.169289 22159 raft_consensus.cc:374] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:58.169343 22159 raft_consensus.cc:729] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95eb3b1d2984a2d8ea11efcc9121a5a, State: Initialized, Role: FOLLOWER
I20260808 07:13:58.169399 22159 consensus_queue.cc:260] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:13:58.169497 22159 ts_tablet_manager.cc:1428] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:13:58.170053 22724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.166019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:43605 (local address 127.0.0.1:43460)
0808 07:13:58.166550 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.166559 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:58.166614 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:58.166792 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:58.166793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:58.166795 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:58.166862 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.166863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.167712 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.167713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.168926 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.168929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.169812 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.169814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.169891 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.169907 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 07:13:58.169955 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:13:58.169993 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":10,"thread_start_us":133,"threads_started":1}
I20260808 07:13:58.170303 22725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.166353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.30.54.66:43605)
0808 07:13:58.166521 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.166526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.166639 (+ 113us) server_negotiation.cc:407] Connection header received
0808 07:13:58.166682 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.166684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.166699 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.166795 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:58.166961 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.167622 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.169002 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.169656 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.169718 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.169776 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:58.170195 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 07:13:58.170237 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260808 07:13:58.170990 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.165920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:46275 (local address 127.0.0.1:37928)
0808 07:13:58.165956 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.165966 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:58.165999 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:58.166379 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:58.166380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:58.166382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:58.166427 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.166429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.167803 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.167804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.169806 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.169808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.170746 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.170747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.170802 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.170804 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:13:58.170906 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:13:58.170946 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":916}
I20260808 07:13:58.171028 22723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.166023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.30.54.67:46275)
0808 07:13:58.166215 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.166220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.166231 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:13:58.166267 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.166269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.166284 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.166369 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:58.166578 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.167683 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.169898 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.170675 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.170745 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.170795 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:58.170921 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 07:13:58.170957 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260808 07:13:58.171089 22423 tablet_service.cc:1467] Processing CreateTablet for tablet 9130591296d94840bbe01d2c4bdc2103 (DEFAULT_TABLE table=test-table [id=a0ee20f0ec7c421c9d2c77314fa411ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:58.171478 22423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9130591296d94840bbe01d2c4bdc2103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:58.174023 22727 tablet_bootstrap.cc:492] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: Bootstrap starting.
I20260808 07:13:58.174423 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 9130591296d94840bbe01d2c4bdc2103 (DEFAULT_TABLE table=test-table [id=a0ee20f0ec7c421c9d2c77314fa411ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:13:58.174613 22727 tablet_bootstrap.cc:654] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:58.174732 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9130591296d94840bbe01d2c4bdc2103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:13:58.175194 22727 log.cc:824] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:58.175966 22727 tablet_bootstrap.cc:492] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: No bootstrap required, opened a new log
I20260808 07:13:58.176021 22727 ts_tablet_manager.cc:1397] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:13:58.177429 22729 tablet_bootstrap.cc:492] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: Bootstrap starting.
I20260808 07:13:58.177450 22727 raft_consensus.cc:348] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:13:58.177645 22727 raft_consensus.cc:374] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:58.177701 22727 raft_consensus.cc:729] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e2c5c39164d423b90acd9279feafce4, State: Initialized, Role: FOLLOWER
I20260808 07:13:58.177850 22727 consensus_queue.cc:260] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:13:58.178134 22727 ts_tablet_manager.cc:1428] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:13:58.178149 22729 tablet_bootstrap.cc:654] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: Neither blocks nor log segments found. Creating new log.
I20260808 07:13:58.178155 22523 heartbeater.cc:503] Master 127.30.54.126:45057 was elected leader, sending a full tablet report...
I20260808 07:13:58.178613 22729 log.cc:824] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: Log is configured to *not* fsync() on all Append() calls
I20260808 07:13:58.179322 22729 tablet_bootstrap.cc:492] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: No bootstrap required, opened a new log
I20260808 07:13:58.179411 22729 ts_tablet_manager.cc:1397] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:13:58.180863 22729 raft_consensus.cc:348] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:13:58.181015 22729 raft_consensus.cc:374] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:13:58.181061 22729 raft_consensus.cc:729] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7cef30da314ee6b15965e49ccc8b42, State: Initialized, Role: FOLLOWER
I20260808 07:13:58.181185 22729 consensus_queue.cc:260] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:13:58.181475 22729 ts_tablet_manager.cc:1428] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:13:58.181488 22714 heartbeater.cc:503] Master 127.30.54.126:45057 was elected leader, sending a full tablet report...
I20260808 07:13:58.200438 22733 raft_consensus.cc:482] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:13:58.200525 22733 raft_consensus.cc:504] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:13:58.200923 22733 leader_election.cc:288] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605), c95eb3b1d2984a2d8ea11efcc9121a5a (127.30.54.65:45561)
I20260808 07:13:58.203915 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.201052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:43605 (local address 127.30.54.67:45587)
0808 07:13:58.201129 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.201141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:58.201187 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:58.201339 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:58.201340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:58.201341 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:58.201409 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.201411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.202037 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.202038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.203010 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.203012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.203725 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.203726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.203785 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.203797 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:13:58.203843 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:13:58.203870 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 07:13:58.203991 22725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.201108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:45587 (local address 127.30.54.66:43605)
0808 07:13:58.201180 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.201182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.201192 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:13:58.201253 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.201254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.201264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.201336 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:58.201491 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.201952 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.203098 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.203642 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.203720 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.203727 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:58.203874 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:13:58.203909 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:13:58.204316 22451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9130591296d94840bbe01d2c4bdc2103" candidate_uuid: "5c7cef30da314ee6b15965e49ccc8b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e2c5c39164d423b90acd9279feafce4" is_pre_election: true
I20260808 07:13:58.204396 22734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.201057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:45561 (local address 127.30.54.67:33799)
0808 07:13:58.201285 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.201323 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:13:58.201405 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:13:58.201555 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:13:58.201556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:13:58.201557 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:13:58.201622 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.201623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.202343 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.202344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.203201 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:13:58.203204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.204117 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.204120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:13:58.204211 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.204215 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:13:58.204279 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:13:58.204316 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":80,"thread_start_us":100,"threads_started":1}
I20260808 07:13:58.204398 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.201128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:33799 (local address 127.30.54.65:45561)
0808 07:13:58.201176 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.201178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.201399 (+ 221us) server_negotiation.cc:407] Connection header received
0808 07:13:58.201463 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.201464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.201486 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.201547 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:13:58.201695 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.202234 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.203298 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.203937 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.204007 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.204082 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:58.204305 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 07:13:58.204351 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:13:58.204474 22451 raft_consensus.cc:2454] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c7cef30da314ee6b15965e49ccc8b42 in term 0.
I20260808 07:13:58.204710 22547 leader_election.cc:302] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7cef30da314ee6b15965e49ccc8b42, 8e2c5c39164d423b90acd9279feafce4; no voters:
I20260808 07:13:58.204701 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9130591296d94840bbe01d2c4bdc2103" candidate_uuid: "5c7cef30da314ee6b15965e49ccc8b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" is_pre_election: true
I20260808 07:13:58.204836 22238 raft_consensus.cc:2454] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c7cef30da314ee6b15965e49ccc8b42 in term 0.
I20260808 07:13:58.204844 22733 raft_consensus.cc:2781] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:13:58.204909 22733 raft_consensus.cc:482] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:13:58.204946 22733 raft_consensus.cc:3037] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:58.205595 22733 raft_consensus.cc:504] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:13:58.205740 22733 leader_election.cc:288] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [CANDIDATE]: Term 1 election: Requested vote from peers 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605), c95eb3b1d2984a2d8ea11efcc9121a5a (127.30.54.65:45561)
I20260808 07:13:58.205883 22238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9130591296d94840bbe01d2c4bdc2103" candidate_uuid: "5c7cef30da314ee6b15965e49ccc8b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a"
I20260808 07:13:58.205955 22451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9130591296d94840bbe01d2c4bdc2103" candidate_uuid: "5c7cef30da314ee6b15965e49ccc8b42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e2c5c39164d423b90acd9279feafce4"
I20260808 07:13:58.205976 22238 raft_consensus.cc:3037] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:58.206028 22451 raft_consensus.cc:3037] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:13:58.206682 22451 raft_consensus.cc:2454] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c7cef30da314ee6b15965e49ccc8b42 in term 1.
I20260808 07:13:58.206688 22238 raft_consensus.cc:2454] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c7cef30da314ee6b15965e49ccc8b42 in term 1.
I20260808 07:13:58.206869 22549 leader_election.cc:302] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7cef30da314ee6b15965e49ccc8b42, c95eb3b1d2984a2d8ea11efcc9121a5a; no voters:
I20260808 07:13:58.206955 22733 raft_consensus.cc:2781] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:13:58.207096 22733 raft_consensus.cc:686] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 1 LEADER]: Becoming Leader. State: Replica: 5c7cef30da314ee6b15965e49ccc8b42, State: Running, Role: LEADER
I20260808 07:13:58.207192 22733 consensus_queue.cc:237] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:13:58.208030 22036 catalog_manager.cc:5495] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67). New cstate: current_term: 1 leader_uuid: "5c7cef30da314ee6b15965e49ccc8b42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } health_report { overall_health: UNKNOWN } } }
I20260808 07:13:58.213994 22723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.211911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37932 (local address 127.30.54.67:46275)
0808 07:13:58.211964 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:13:58.211966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:13:58.211975 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:13:58.212002 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:13:58.212003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:13:58.212026 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:13:58.212084 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:13:58.212224 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.212643 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.213325 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:13:58.213494 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:13:58.213541 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:13:58.213717 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:13:58.213802 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:13:58.213882 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:13:58.213915 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 07:13:58.213932 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:13:58.215617 22451 raft_consensus.cc:1261] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [term 1 FOLLOWER]: Refusing update from remote peer 5c7cef30da314ee6b15965e49ccc8b42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:58.215621 22238 raft_consensus.cc:1261] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Refusing update from remote peer 5c7cef30da314ee6b15965e49ccc8b42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:13:58.215929 22733 consensus_queue.cc:1035] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:58.216042 22735 consensus_queue.cc:1035] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:13:58.217998 22738 mvcc.cc:205] Tried to move back new op lower bound from 7316165583728930816 to 7316165583697563648. Current Snapshot: MvccSnapshot[applied={T|T < 7316165583728930816}]
I20260808 07:13:58.218318 22739 mvcc.cc:205] Tried to move back new op lower bound from 7316165583728930816 to 7316165583697563648. Current Snapshot: MvccSnapshot[applied={T|T < 7316165583728930816}]
I20260808 07:13:58.218892 22741 mvcc.cc:205] Tried to move back new op lower bound from 7316165583728930816 to 7316165583697563648. Current Snapshot: MvccSnapshot[applied={T|T < 7316165583728930816}]
I20260808 07:13:58.220492 22036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:58.220582 22036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:13:58.221225 22036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Sending DeleteTablet for 3 replicas of tablet 9130591296d94840bbe01d2c4bdc2103
I20260808 07:13:58.221549 22219 tablet_service.cc:1514] Processing DeleteTablet for tablet 9130591296d94840bbe01d2c4bdc2103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:58 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:50614
I20260808 07:13:58.221556 22613 tablet_service.cc:1514] Processing DeleteTablet for tablet 9130591296d94840bbe01d2c4bdc2103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:58 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:37928
I20260808 07:13:58.221587 22423 tablet_service.cc:1514] Processing DeleteTablet for tablet 9130591296d94840bbe01d2c4bdc2103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:13:58 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:43460
I20260808 07:13:58.221648 22325 tablet_replica.cc:330] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: stopping tablet replica
I20260808 07:13:58.221701 22325 raft_consensus.cc:2229] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:58.221763 22325 raft_consensus.cc:2258] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:58.221799 22756 tablet_replica.cc:330] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: stopping tablet replica
I20260808 07:13:58.221897 22756 raft_consensus.cc:2229] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:13:58.221994 22756 raft_consensus.cc:2258] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:58.221985 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 22096
I20260808 07:13:58.225736 22756 ts_tablet_manager.cc:1905] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:58.227761 22756 ts_tablet_manager.cc:1918] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:58.227836 22756 log.cc:1196] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/9130591296d94840bbe01d2c4bdc2103
I20260808 07:13:58.227771 22754 tablet_replica.cc:330] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: stopping tablet replica
I20260808 07:13:58.228096 22756 ts_tablet_manager.cc:1939] T 9130591296d94840bbe01d2c4bdc2103 P 8e2c5c39164d423b90acd9279feafce4: Deleting consensus metadata
I20260808 07:13:58.228116 22754 raft_consensus.cc:2229] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:13:58.228353 22754 raft_consensus.cc:2258] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:13:58.228468 22016 catalog_manager.cc:4841] TS 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605): tablet 9130591296d94840bbe01d2c4bdc2103 (table test-table [id=a0ee20f0ec7c421c9d2c77314fa411ca]) successfully deleted
I20260808 07:13:58.230670 22754 ts_tablet_manager.cc:1905] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:13:58.232715 22754 ts_tablet_manager.cc:1918] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:13:58.232889 22754 log.cc:1196] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/9130591296d94840bbe01d2c4bdc2103
I20260808 07:13:58.233757 22754 ts_tablet_manager.cc:1939] T 9130591296d94840bbe01d2c4bdc2103 P 5c7cef30da314ee6b15965e49ccc8b42: Deleting consensus metadata
I20260808 07:13:58.234542 22016 catalog_manager.cc:4841] TS 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275): tablet 9130591296d94840bbe01d2c4bdc2103 (table test-table [id=a0ee20f0ec7c421c9d2c77314fa411ca]) successfully deleted
W20260808 07:13:58.242022 22017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:45561 (error 108)
I20260808 07:13:58.242341 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 22320
I20260808 07:13:58.247206 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:13:58.242444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:45561 (local address 127.0.0.1:50620)
0808 07:13:58.247113 (+ 4669us) negotiation.cc:107] Waiting for socket to connect
0808 07:13:58.247164 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:45561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4643}
W20260808 07:13:58.255888 22017 catalog_manager.cc:4584] TS c95eb3b1d2984a2d8ea11efcc9121a5a (127.30.54.65:45561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9130591296d94840bbe01d2c4bdc2103: Network error: Client connection negotiation failed: client connection to 127.30.54.65:45561: connect: Connection refused (error 111)
I20260808 07:13:58.260723 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 22526
I20260808 07:13:58.269801 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 21994
I20260808 07:14:18.281379 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45057
--webserver_interface=127.30.54.126
--webserver_port=35697
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:45057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:18.363497 22761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:18.363737 22761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:18.363786 22761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:18.364935 22761 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:14:18.365006 22761 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:14:18.365027 22761 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:14:18.365046 22761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:18.365073 22761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:18.365103 22761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:18.365126 22761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:18.365154 22761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:18.366639 22761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:45057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:45057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=35697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:18.366935 22761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:18.367166 22761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:18.372409 22769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:18.372412 22773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:18.372485 22761 server_base.cc:1034] running on GCE node
W20260808 07:14:18.372442 22766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:18.372929 22761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:18.373208 22761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:18.374414 22761 hybrid_clock.cc:648] HybridClock initialized: now 1786173258374411 us; error 40 us; skew 500 ppm
Aug 08 07:14:18 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:18.376905 22761 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:14:18.377362 22761 webserver.cc:460] Webserver started at http://127.30.54.126:35697/ using document root <none> and password file <none>
I20260808 07:14:18.377591 22761 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:18.377658 22761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:18.378988 22761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:18.379639 22782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.379801 22761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.379906 22761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "f95fefbd014d42848005fef40e68c7a1"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "2dda25621c6ffec6a99d560b5a38fb9c"
server_key_iv: "5ccb0156ba3de9917e046a811d717687"
server_key_version: "encryptionkey@0"
I20260808 07:14:18.380219 22761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:18.405187 22761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:18.407114 22761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:18.407275 22761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:18.411410 22761 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:45057
I20260808 07:14:18.411448 22868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:45057 every 8 connection(s)
I20260808 07:14:18.411842 22761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:18.413304 22869 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:14:18.413976 22869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Bootstrap starting.
I20260808 07:14:18.417421 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 22761
I20260808 07:14:18.417727 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:45561
--local_ip_for_outbound_sockets=127.30.54.65
--tserver_master_addrs=127.30.54.126:45057
--webserver_port=33819
--webserver_interface=127.30.54.65
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:14:18.422926 22869 log.cc:824] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:18.425256 22869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:14:18.425577 22869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: Bootstrap complete.
I20260808 07:14:18.428081 22869 raft_consensus.cc:348] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:14:18.428328 22869 raft_consensus.cc:729] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95fefbd014d42848005fef40e68c7a1, State: Initialized, Role: FOLLOWER
I20260808 07:14:18.428519 22869 consensus_queue.cc:260] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:14:18.428601 22869 raft_consensus.cc:388] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:18.428655 22869 raft_consensus.cc:482] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:18.428707 22869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:14:18.429698 22869 raft_consensus.cc:504] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:14:18.429819 22869 leader_election.cc:302] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f95fefbd014d42848005fef40e68c7a1; no voters:
I20260808 07:14:18.430025 22869 leader_election.cc:288] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:14:18.430061 22876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:14:18.430183 22876 raft_consensus.cc:686] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [term 3 LEADER]: Becoming Leader. State: Replica: f95fefbd014d42848005fef40e68c7a1, State: Running, Role: LEADER
I20260808 07:14:18.430306 22869 sys_catalog.cc:564] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:18.430334 22876 consensus_queue.cc:237] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } }
I20260808 07:14:18.430703 22877 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f95fefbd014d42848005fef40e68c7a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } } }
I20260808 07:14:18.430795 22877 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:18.431048 22878 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f95fefbd014d42848005fef40e68c7a1. Latest consensus state: current_term: 3 leader_uuid: "f95fefbd014d42848005fef40e68c7a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95fefbd014d42848005fef40e68c7a1" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 45057 } } }
I20260808 07:14:18.431156 22878 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:18.432200 22904 catalog_manager.cc:1259] Loaded cluster ID: 66c7805905a44ca2ba34b6f634bdd77f
I20260808 07:14:18.432264 22904 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: loading cluster ID for follower catalog manager: success
I20260808 07:14:18.433094 22904 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f95fefbd014d42848005fef40e68c7a1: acquiring CA information for follower catalog manager: success
F20260808 07:14:18.433449 22904 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2fe88679ad google::LogMessage::Fail() at ??:0
@ 0x7f2fe886a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f2fe88674de google::LogMessage::Flush() at ??:0
@ 0x7f2fe8868a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2feafd4104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2ff07c06b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f2ff07c0989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f2ff07c0e4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f2ff07c38c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f2fea72a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2feb58e6db start_thread at ??:0
@ 0x7f2fe736561f clone at ??:0
W20260808 07:14:18.512945 22874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:18.513136 22874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:18.513159 22874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:18.514281 22874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:18.514320 22874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:18.514346 22874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:14:18.515815 22874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:45561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=33819
--tserver_master_addrs=127.30.54.126:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:18.516045 22874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:18.516288 22874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:18.521377 22928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:18.521368 22929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:18.521476 22874 server_base.cc:1034] running on GCE node
W20260808 07:14:18.521561 22931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:18.522029 22874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:18.522277 22874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:18.523437 22874 hybrid_clock.cc:648] HybridClock initialized: now 1786173258523420 us; error 34 us; skew 500 ppm
Aug 08 07:14:18 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:18.525976 22874 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:14:18.526417 22874 webserver.cc:460] Webserver started at http://127.30.54.65:33819/ using document root <none> and password file <none>
I20260808 07:14:18.526640 22874 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:18.526711 22874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:18.528092 22874 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:18.528853 22943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.529043 22874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:18.529150 22874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "539f23a1aba322f66263296ee3bf9f26"
server_key_iv: "e7d45c20134d20a54d2aa4bf830897cd"
server_key_version: "encryptionkey@0"
I20260808 07:14:18.529474 22874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:18.536908 22874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:18.539139 22874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:18.539290 22874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:18.539515 22874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:18.539932 22952 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:14:18.541788 22874 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:14:18.541846 22874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:14:18.541906 22874 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:14:18.542481 22874 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:14:18.542536 22874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:14:18.542603 22952 tablet_bootstrap.cc:492] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Bootstrap starting.
I20260808 07:14:18.549912 22874 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:45561
I20260808 07:14:18.550280 22874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:18.550761 23114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:45561 every 8 connection(s)
I20260808 07:14:18.553363 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 22874
I20260808 07:14:18.553421 22952 log.cc:824] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:18.553720 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:43605
--local_ip_for_outbound_sockets=127.30.54.66
--tserver_master_addrs=127.30.54.126:45057
--webserver_port=38549
--webserver_interface=127.30.54.66
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:14:18.556164 22952 tablet_bootstrap.cc:492] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:14:18.556488 22952 tablet_bootstrap.cc:492] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Bootstrap complete.
I20260808 07:14:18.557741 22952 ts_tablet_manager.cc:1397] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.009s
I20260808 07:14:18.559202 22952 raft_consensus.cc:348] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:14:18.559561 22952 raft_consensus.cc:729] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95eb3b1d2984a2d8ea11efcc9121a5a, State: Initialized, Role: FOLLOWER
I20260808 07:14:18.559746 22952 consensus_queue.cc:260] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:14:18.560037 22952 ts_tablet_manager.cc:1428] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260808 07:14:18.625277 23118 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:14:18.551034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:46171)
0808 07:14:18.551194 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:18.551236 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:18.551662 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:18.625083 (+ 73421us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
W20260808 07:14:18.625560 23115 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:45057 (0 consecutive failures): Network error: Failed to ping master at 127.30.54.126:45057: Client connection negotiation failed: client connection to 127.30.54.126:45057: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:14:18.625900 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:18.625784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:47275)
0808 07:14:18.625838 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:18.625879 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:14:18.626241 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:18.626163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:55163)
0808 07:14:18.626206 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:18.626227 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:14:18.626364 23115 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:14:18.637593 23122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:18.637775 23122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:18.637799 23122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:18.638912 23122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:18.638949 23122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:18.638983 23122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:14:18.640378 23122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:43605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=38549
--tserver_master_addrs=127.30.54.126:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:18.640563 23122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:18.640723 23122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:18.645872 23131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:18.645910 23133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:18.645896 23136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:18.646255 23122 server_base.cc:1034] running on GCE node
I20260808 07:14:18.646585 23122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:18.646911 23122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:18.648084 23122 hybrid_clock.cc:648] HybridClock initialized: now 1786173258648072 us; error 26 us; skew 500 ppm
Aug 08 07:14:18 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:18.650699 23122 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:14:18.651201 23122 webserver.cc:460] Webserver started at http://127.30.54.66:38549/ using document root <none> and password file <none>
I20260808 07:14:18.651516 23122 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:18.651599 23122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:18.653086 23122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:14:18.653932 23145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.654183 23122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:18.654320 23122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "8e2c5c39164d423b90acd9279feafce4"
format_stamp: "Formatted at 2026-08-08 07:13:36 on dist-test-slave-znh6"
server_key: "122f25eea7d38f35e619b916defc5527"
server_key_iv: "5a681800ceccc4e9a91e601dabe39579"
server_key_version: "encryptionkey@0"
I20260808 07:14:18.654662 23122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:18.680088 23122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:18.681830 23122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:18.682003 23122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:18.682236 23122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:18.682554 23122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:18.682607 23122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.682644 23122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:18.682672 23122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.688913 23122 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:43605
I20260808 07:14:18.688973 23314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:43605 every 8 connection(s)
I20260808 07:14:18.689296 23122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:14:18.690246 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:18.689879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:53625)
0808 07:14:18.690040 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:18.690109 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
W20260808 07:14:18.690642 23315 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:45057 (0 consecutive failures): Network error: Failed to ping master at 127.30.54.126:45057: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
I20260808 07:14:18.690939 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:18.690836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:36649)
0808 07:14:18.690885 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:18.690921 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:14:18.691215 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:18.691139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:47223)
0808 07:14:18.691181 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:18.691202 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:14:18.691351 23315 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:14:18.691955 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 23122
I20260808 07:14:18.692229 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:46275
--local_ip_for_outbound_sockets=127.30.54.67
--tserver_master_addrs=127.30.54.126:45057
--webserver_port=45279
--webserver_interface=127.30.54.67
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:18.773972 23319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:18.774169 23319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:18.774194 23319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:18.775348 23319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:18.775391 23319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:18.775417 23319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:14:18.776851 23319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:46275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=45279
--tserver_master_addrs=127.30.54.126:45057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:18.777086 23319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:18.777313 23319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:18.782523 23330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:18.782567 23319 server_base.cc:1034] running on GCE node
W20260808 07:14:18.782508 23326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:18.782495 23325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:18.782903 23319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:18.783146 23319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:18.784325 23319 hybrid_clock.cc:648] HybridClock initialized: now 1786173258784321 us; error 35 us; skew 500 ppm
Aug 08 07:14:18 dist-test-slave-znh6 krb5kdc[21182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:18.786860 23319 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:14:18.787298 23319 webserver.cc:460] Webserver started at http://127.30.54.67:45279/ using document root <none> and password file <none>
I20260808 07:14:18.787528 23319 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:18.787596 23319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:18.788887 23319 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:18.789464 23342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.789644 23319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.789748 23319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "5c7cef30da314ee6b15965e49ccc8b42"
format_stamp: "Formatted at 2026-08-08 07:13:37 on dist-test-slave-znh6"
server_key: "a2e052000f111ba9830f9dae30f35864"
server_key_iv: "bfa326ce964697888bea5da63251a2a7"
server_key_version: "encryptionkey@0"
I20260808 07:14:18.790086 23319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:18.814584 23319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:18.816581 23319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:18.816747 23319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:18.816990 23319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:18.817313 23319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:18.817368 23319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.817404 23319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:18.817432 23319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:18.823593 23319 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:46275
I20260808 07:14:18.823645 23509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:46275 every 8 connection(s)
I20260808 07:14:18.823982 23319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:14:18.825073 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:18.824707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:41987)
0808 07:14:18.824864 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:18.824937 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
W20260808 07:14:18.825486 23510 heartbeater.cc:644] Failed to heartbeat to 127.30.54.126:45057 (0 consecutive failures): Network error: Failed to ping master at 127.30.54.126:45057: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
I20260808 07:14:18.825781 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:18.825683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:37645)
0808 07:14:18.825729 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:18.825757 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:14:18.826058 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:18.825971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:44173)
0808 07:14:18.826015 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:18.826038 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:14:18.826170 23510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:14:18.827879 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 23319
I20260808 07:14:19.626972 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:19.626701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:58183)
0808 07:14:19.626861 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:19.626905 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260808 07:14:19.691843 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:19.691624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:49255)
0808 07:14:19.691754 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:19.691796 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260808 07:14:19.760905 23520 raft_consensus.cc:482] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:19.761062 23520 raft_consensus.cc:504] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:14:19.761395 23520 leader_election.cc:288] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275), 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605)
Aug 08 07:14:19 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.67@KRBTEST.COM
Aug 08 07:14:19 dist-test-slave-znh6 krb5kdc[21182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.66@KRBTEST.COM
I20260808 07:14:19.772292 23525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:19.761842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:58665 (local address 127.30.54.66:43605)
0808 07:14:19.762010 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:19.762018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:19.762045 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:14:19.762466 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:19.762468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:19.762502 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:19.762662 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:19.763426 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:19.764281 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:19.765290 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:19.765690 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:19.765806 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:19.768713 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:19.768716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:19.768716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:19.768725 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:19.770580 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:19.770965 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:19.770968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:19.770968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:19.771075 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:19.771272 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:19.771273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:19.771273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:19.771568 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:19.771656 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:19.772042 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 07:14:19.772176 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260808 07:14:19.772292 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:19.761695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:46281 (local address 127.30.54.67:46275)
0808 07:14:19.761906 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:19.761919 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:19.761945 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:14:19.762382 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:19.762384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:19.762452 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:19.762563 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:19.763450 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:19.764298 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:19.765277 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:19.765705 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:19.765831 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:19.768266 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:19.768269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:19.768269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:19.768279 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:19.770341 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:19.770799 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:19.770801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:19.770802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:19.770969 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:19.771263 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:19.771265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:19.771265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:19.771568 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:19.771656 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:19.772041 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 07:14:19.772178 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":83,"threads_started":1}
I20260808 07:14:19.772295 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:19.761535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:46275 (local address 127.30.54.65:46281)
0808 07:14:19.761578 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:19.761593 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:19.761670 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:19.762646 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:19.762648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:19.763103 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:19.763294 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:19.763300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:19.764418 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:19.764419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:19.765184 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:19.765186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:19.765813 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:19.765814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:19.765899 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:19.766536 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:19.766540 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:19.768165 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:19.770437 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:19.770443 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:19.770445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:19.770717 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:19.771042 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:19.771043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:19.771043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:19.771187 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:19.771655 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:19.771664 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:19.771836 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 07:14:19.771960 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 07:14:19.772155 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":12}
I20260808 07:14:19.772353 23521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:19.761673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:43605 (local address 127.30.54.65:58665)
0808 07:14:19.761788 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:19.761798 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:19.761867 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:19.762670 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:19.762671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:19.763101 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:19.763294 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:19.763300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:19.764384 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:19.764385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:19.765185 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:19.765188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:19.765777 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:19.765778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:19.765888 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:19.766472 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:19.766480 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:19.768614 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:19.770658 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:19.770659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:19.770659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:19.770896 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:19.771138 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:19.771139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:19.771139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:19.771194 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:19.771655 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:19.771664 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:19.771829 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 07:14:19.771960 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 07:14:19.772211 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":184,"thread_start_us":65,"threads_started":1}
W20260808 07:14:19.773200 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
W20260808 07:14:19.773518 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
I20260808 07:14:19.773566 22945 leader_election.cc:302] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95eb3b1d2984a2d8ea11efcc9121a5a; no voters: 5c7cef30da314ee6b15965e49ccc8b42, 8e2c5c39164d423b90acd9279feafce4
I20260808 07:14:19.773765 23520 raft_consensus.cc:2726] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:19.826771 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:19.826517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:42063)
0808 07:14:19.826699 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:19.826736 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":110,"threads_started":1}
I20260808 07:14:20.627701 23528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:20.627482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:34561)
0808 07:14:20.627612 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:20.627652 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260808 07:14:20.693930 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:20.693735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:44689)
0808 07:14:20.693848 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:20.693884 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
I20260808 07:14:20.827526 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:20.827283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:48405)
0808 07:14:20.827433 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:20.827467 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260808 07:14:21.602540 23533 raft_consensus.cc:482] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:21.602654 23533 raft_consensus.cc:504] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:14:21.602880 23533 leader_election.cc:288] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275), 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605)
W20260808 07:14:21.603308 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
W20260808 07:14:21.603420 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
I20260808 07:14:21.603449 22945 leader_election.cc:302] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95eb3b1d2984a2d8ea11efcc9121a5a; no voters: 5c7cef30da314ee6b15965e49ccc8b42, 8e2c5c39164d423b90acd9279feafce4
I20260808 07:14:21.603539 23533 raft_consensus.cc:2726] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:21.628337 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:21.628145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:58365)
0808 07:14:21.628264 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:21.628296 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260808 07:14:21.694555 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:21.694349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:50325)
0808 07:14:21.694484 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:21.694519 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260808 07:14:21.828137 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:21.827943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:32893)
0808 07:14:21.828062 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:21.828099 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260808 07:14:22.629037 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:22.628815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:39381)
0808 07:14:22.628944 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:22.628987 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260808 07:14:22.695214 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:22.695006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:49621)
0808 07:14:22.695140 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:22.695175 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260808 07:14:22.828787 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:22.828566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:46619)
0808 07:14:22.828721 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:22.828748 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":106,"threads_started":1}
I20260808 07:14:23.629838 23543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:23.629528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:34983)
0808 07:14:23.629755 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:23.629792 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":81,"threads_started":1}
I20260808 07:14:23.695838 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:23.695639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:59757)
0808 07:14:23.695770 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:23.695803 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260808 07:14:23.724563 23547 raft_consensus.cc:482] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:23.724665 23547 raft_consensus.cc:504] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:14:23.724859 23547 leader_election.cc:288] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275), 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605)
W20260808 07:14:23.725277 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
W20260808 07:14:23.725359 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
I20260808 07:14:23.725379 22945 leader_election.cc:302] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95eb3b1d2984a2d8ea11efcc9121a5a; no voters: 5c7cef30da314ee6b15965e49ccc8b42, 8e2c5c39164d423b90acd9279feafce4
I20260808 07:14:23.725457 23547 raft_consensus.cc:2726] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:23.829450 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:23.829242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:33905)
0808 07:14:23.829363 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:23.829399 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260808 07:14:24.630563 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:24.630325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:39707)
0808 07:14:24.630476 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:24.630513 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260808 07:14:24.696487 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:24.696270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:40829)
0808 07:14:24.696408 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:24.696443 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260808 07:14:24.830085 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:24.829870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:49033)
0808 07:14:24.830005 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:24.830041 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260808 07:14:25.631287 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:25.631062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:45951)
0808 07:14:25.631202 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:25.631240 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260808 07:14:25.697098 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:25.696871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:35679)
0808 07:14:25.697014 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:25.697060 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260808 07:14:25.830780 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:25.830555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:36797)
0808 07:14:25.830684 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:25.830716 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260808 07:14:26.579684 23561 raft_consensus.cc:482] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:26.579779 23561 raft_consensus.cc:504] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:14:26.579936 23561 leader_election.cc:288] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275), 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605)
W20260808 07:14:26.580358 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
W20260808 07:14:26.580446 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
I20260808 07:14:26.580467 22945 leader_election.cc:302] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95eb3b1d2984a2d8ea11efcc9121a5a; no voters: 5c7cef30da314ee6b15965e49ccc8b42, 8e2c5c39164d423b90acd9279feafce4
I20260808 07:14:26.580564 23561 raft_consensus.cc:2726] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:26.631951 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:26.631758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:55033)
0808 07:14:26.631873 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:26.631905 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260808 07:14:26.697762 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:26.697555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:37359)
0808 07:14:26.697683 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:26.697712 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260808 07:14:26.831472 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:26.831260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:42591)
0808 07:14:26.831373 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:26.831405 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260808 07:14:27.632622 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:27.632400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:55099)
0808 07:14:27.632535 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:27.632573 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260808 07:14:27.698401 23570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:27.698200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:46781)
0808 07:14:27.698325 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:27.698361 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260808 07:14:27.832086 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:27.831899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:36647)
0808 07:14:27.832014 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:27.832048 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260808 07:14:28.633348 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:28.633064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:48009)
0808 07:14:28.633200 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:28.633288 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260808 07:14:28.699129 23574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:28.698915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:51745)
0808 07:14:28.699056 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:28.699090 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260808 07:14:28.832758 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:28.832495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:43701)
0808 07:14:28.832635 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:28.832699 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260808 07:14:29.634023 23577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.633773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:48709)
0808 07:14:29.633938 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.633976 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":82,"threads_started":1}
I20260808 07:14:29.699819 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.699617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:44713)
0808 07:14:29.699739 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.699777 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260808 07:14:29.833438 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:29.833192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:34367)
0808 07:14:29.833340 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:29.833375 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260808 07:14:30.634704 23584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:30.634488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:42281)
0808 07:14:30.634618 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:30.634655 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260808 07:14:30.700534 23587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:30.700327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:56613)
0808 07:14:30.700451 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:30.700493 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260808 07:14:30.834110 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:30.833901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:45569)
0808 07:14:30.834028 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:30.834064 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260808 07:14:31.635421 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:31.635198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:51323)
0808 07:14:31.635331 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:31.635369 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260808 07:14:31.701293 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:31.701057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:52163)
0808 07:14:31.701203 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:31.701246 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260808 07:14:31.834882 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:31.834586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:38387)
0808 07:14:31.834733 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:31.834769 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
I20260808 07:14:32.636154 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:32.635928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:40407)
0808 07:14:32.636072 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:32.636103 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260808 07:14:32.702114 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:32.701888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:39411)
0808 07:14:32.702033 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:32.702071 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260808 07:14:32.835554 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:32.835333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:54943)
0808 07:14:32.835474 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:32.835508 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260808 07:14:32.876153 23597 raft_consensus.cc:482] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:32.876277 23597 raft_consensus.cc:504] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7cef30da314ee6b15965e49ccc8b42" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 46275 } } peers { permanent_uuid: "8e2c5c39164d423b90acd9279feafce4" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 43605 } } peers { permanent_uuid: "c95eb3b1d2984a2d8ea11efcc9121a5a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45561 } }
I20260808 07:14:32.876487 23597 leader_election.cc:288] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275), 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605)
W20260808 07:14:32.876883 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e2c5c39164d423b90acd9279feafce4 (127.30.54.66:43605): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
W20260808 07:14:32.876964 22945 leader_election.cc:341] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c7cef30da314ee6b15965e49ccc8b42 (127.30.54.67:46275): Not found: Tablet not found: 9130591296d94840bbe01d2c4bdc2103
I20260808 07:14:32.877005 22945 leader_election.cc:302] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95eb3b1d2984a2d8ea11efcc9121a5a; no voters: 5c7cef30da314ee6b15965e49ccc8b42, 8e2c5c39164d423b90acd9279feafce4
I20260808 07:14:32.877100 23597 raft_consensus.cc:2726] T 9130591296d94840bbe01d2c4bdc2103 P c95eb3b1d2984a2d8ea11efcc9121a5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:14:33.636835 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:33.636602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.65:35389)
0808 07:14:33.636747 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:33.636786 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260808 07:14:33.702868 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:33.702634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.66:38369)
0808 07:14:33.702751 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:33.702782 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260808 07:14:33.836324 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:33.836101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:45057 (local address 127.30.54.67:36149)
0808 07:14:33.836239 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:33.836273 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.126:45057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260808 07:14:33.837154 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 22874
I20260808 07:14:33.847592 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 23122
I20260808 07:14:33.856204 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 23319
2026-08-08T07:14:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58681 ms)
[----------] 30 tests from SecurityITest (264347 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:33 dist-test-slave-znh6 krb5kdc[23607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:33 dist-test-slave-znh6 krb5kdc[23607](info): set up 2 sockets
Aug 08 07:14:33 dist-test-slave-znh6 krb5kdc[23607](info): commencing operation
krb5kdc: starting...
W20260808 07:14:35.080155 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:35 dist-test-slave-znh6 krb5kdc[23607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:14:35.198776 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:36053
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:36053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:35.280273 23629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:35.280503 23629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:35.280541 23629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:35.281657 23629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:35.281729 23629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:35.281754 23629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:35.281776 23629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:35.281805 23629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:35.283397 23629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:36053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:36053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:35.283671 23629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:35.283907 23629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:35.289616 23640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:35.289620 23643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:35.289609 23638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:35.289644 23629 server_base.cc:1034] running on GCE node
I20260808 07:14:35.290174 23629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:35.290469 23629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:35.291646 23629 hybrid_clock.cc:648] HybridClock initialized: now 1786173275291631 us; error 29 us; skew 500 ppm
Aug 08 07:14:35 dist-test-slave-znh6 krb5kdc[23607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:35.294210 23629 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:14:35.294672 23629 webserver.cc:460] Webserver started at http://127.30.54.126:38637/ using document root <none> and password file <none>
I20260808 07:14:35.294934 23629 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:35.294982 23629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:35.295077 23629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:35.296067 23629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "ef8c63498f8a446cb33716093c09e641"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "6a465a93730318846eba331f6a0164d6"
server_key_iv: "ea1e0c34acc309ab307cc25a98c5c48f"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.296365 23629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "ef8c63498f8a446cb33716093c09e641"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "6a465a93730318846eba331f6a0164d6"
server_key_iv: "ea1e0c34acc309ab307cc25a98c5c48f"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.297657 23629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:14:35.298365 23651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.298543 23629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:14:35.298636 23629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "ef8c63498f8a446cb33716093c09e641"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "6a465a93730318846eba331f6a0164d6"
server_key_iv: "ea1e0c34acc309ab307cc25a98c5c48f"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.298707 23629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:35.306193 23629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:35.308475 23629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:35.308640 23629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:35.312575 23629 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:36053
I20260808 07:14:35.312674 23737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:36053 every 8 connection(s)
I20260808 07:14:35.312976 23629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:35.313285 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 23629
I20260808 07:14:35.313426 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:35.313700 30937 external_mini_cluster.cc:1428] Setting key 406c70b9592932ae44901935402b4efc
I20260808 07:14:35.314091 23738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:35.317396 23738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641: Bootstrap starting.
I20260808 07:14:35.318084 23738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:35.318409 23738 log.cc:824] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:35.319288 23738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641: No bootstrap required, opened a new log
Aug 08 07:14:35 dist-test-slave-znh6 krb5kdc[23607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:35.321791 23738 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8c63498f8a446cb33716093c09e641" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36053 } }
I20260808 07:14:35.321990 23738 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:35.322054 23738 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8c63498f8a446cb33716093c09e641, State: Initialized, Role: FOLLOWER
I20260808 07:14:35.322212 23738 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8c63498f8a446cb33716093c09e641" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36053 } }
I20260808 07:14:35.322288 23738 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:35.322310 23738 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:35.322371 23738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:35.322965 23738 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8c63498f8a446cb33716093c09e641" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36053 } }
I20260808 07:14:35.323061 23738 leader_election.cc:302] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef8c63498f8a446cb33716093c09e641; no voters:
I20260808 07:14:35.323266 23738 leader_election.cc:288] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:35.323300 23744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:35.323410 23744 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [term 1 LEADER]: Becoming Leader. State: Replica: ef8c63498f8a446cb33716093c09e641, State: Running, Role: LEADER
I20260808 07:14:35.323555 23738 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:35.323557 23744 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8c63498f8a446cb33716093c09e641" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36053 } }
I20260808 07:14:35.324031 23745 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef8c63498f8a446cb33716093c09e641. Latest consensus state: current_term: 1 leader_uuid: "ef8c63498f8a446cb33716093c09e641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8c63498f8a446cb33716093c09e641" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36053 } } }
I20260808 07:14:35.324126 23745 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:35.324375 23749 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef8c63498f8a446cb33716093c09e641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8c63498f8a446cb33716093c09e641" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 36053 } } }
I20260808 07:14:35.324786 23749 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:35.324970 23757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:35.325225 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.314227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.30.54.126:36053)
0808 07:14:35.314407 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.314414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.314440 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:14:35.314916 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.314928 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.314975 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.315085 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:35.315889 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.316777 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.317880 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.318321 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.318399 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.321058 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:35.321064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:35.321065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:35.321077 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:35.323020 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.323493 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.323496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.323496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.323599 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.323781 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.323781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.323782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.324074 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:35.324213 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.324776 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.324927 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260808 07:14:35.325544 23757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:35.327030 23757 catalog_manager.cc:1347] Generated new cluster ID: 49ee5e79a58d44b2a6fc134e3ded5910
I20260808 07:14:35.327082 23757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:35.341284 23757 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:35.341792 23757 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:35.347765 23757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641: Generated new TSK 0
I20260808 07:14:35.347959 23757 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:14:35.381479 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36053
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:35.463178 23777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:35.463400 23777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:35.463424 23777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:35.464493 23777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:35.464525 23777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:35.464551 23777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:14:35.465969 23777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:35.466144 23777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:35.466318 23777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:35.471503 23784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:35.471500 23777 server_base.cc:1034] running on GCE node
W20260808 07:14:35.471462 23790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:35.471413 23785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:35.471952 23777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:35.472208 23777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:35.473376 23777 hybrid_clock.cc:648] HybridClock initialized: now 1786173275473354 us; error 35 us; skew 500 ppm
Aug 08 07:14:35 dist-test-slave-znh6 krb5kdc[23607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:35.475958 23777 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:14:35.476454 23777 webserver.cc:460] Webserver started at http://127.30.54.65:41411/ using document root <none> and password file <none>
I20260808 07:14:35.476693 23777 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:35.476763 23777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:35.476902 23777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:35.477921 23777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "9dee6b3edaa640ec85945813f7a1f0f5"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "e3e7d35f699a85b901f95f3b1e4585f5"
server_key_iv: "cae8bcd5da96537c7c81091980e596c1"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.478307 23777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "9dee6b3edaa640ec85945813f7a1f0f5"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "e3e7d35f699a85b901f95f3b1e4585f5"
server_key_iv: "cae8bcd5da96537c7c81091980e596c1"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.479765 23777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:14:35.480576 23799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.480764 23777 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:35.480859 23777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "9dee6b3edaa640ec85945813f7a1f0f5"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "e3e7d35f699a85b901f95f3b1e4585f5"
server_key_iv: "cae8bcd5da96537c7c81091980e596c1"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.480912 23777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:35.491708 23777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:35.493904 23777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:35.494011 23777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:35.494175 23777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:35.494480 23777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:35.494515 23777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.494536 23777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:35.494587 23777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.500442 23777 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:40383
I20260808 07:14:35.500524 23964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:40383 every 8 connection(s)
I20260808 07:14:35.500833 23777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:35.506337 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 23777
I20260808 07:14:35.506431 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
Aug 08 07:14:35 dist-test-slave-znh6 krb5kdc[23607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:35.506652 30937 external_mini_cluster.cc:1428] Setting key c9cdf97543b0af932bd37511346fafdf
I20260808 07:14:35.510016 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.501549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:51187 (local address 127.30.54.126:36053)
0808 07:14:35.501596 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.501598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.501838 (+ 240us) server_negotiation.cc:407] Connection header received
0808 07:14:35.502336 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.502337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.502346 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.502399 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:35.503388 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.503895 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.504841 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.505045 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.505099 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.507171 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:35.507172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:35.507173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:35.507180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:35.508557 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.509036 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.509037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.509038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.509103 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.509399 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.509400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.509400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.509509 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:35.509561 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.509878 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.509945 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:14:35.510317 23968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.501534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36053 (local address 127.30.54.65:51187)
0808 07:14:35.501691 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:35.501725 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:35.502245 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:35.502533 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:35.502537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:35.503109 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:35.503300 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.503304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.503986 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.503987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.504766 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.504768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.505122 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.505123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.505218 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.505616 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:35.505624 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:35.507065 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:35.508658 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:35.508665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:35.508667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:35.508946 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:35.509185 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:35.509186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:35.509187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:35.509331 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:35.509575 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:35.509581 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:35.509736 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 07:14:35.509855 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:14:35.510028 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260808 07:14:35.510897 23966 heartbeater.cc:347] Connected to a master server at 127.30.54.126:36053
I20260808 07:14:35.511009 23966 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:35.511180 23966 heartbeater.cc:511] Master 127.30.54.126:36053 requested a full tablet report, sending...
I20260808 07:14:35.511715 23674 ts_manager.cc:194] Registered new tserver with Master: 9dee6b3edaa640ec85945813f7a1f0f5 (127.30.54.65:40383)
I20260808 07:14:35.512398 23674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:51187
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:14:35.534582 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36053
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:35.615141 23972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:35.615341 23972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:35.615365 23972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:35.616484 23972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:35.616528 23972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:35.616554 23972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:14:35.617975 23972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:35.618219 23972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:35.618451 23972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:35.623596 23980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:35.623646 23972 server_base.cc:1034] running on GCE node
W20260808 07:14:35.623613 23981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:35.623611 23988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:35.624042 23972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:35.624285 23972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:35.625520 23972 hybrid_clock.cc:648] HybridClock initialized: now 1786173275625505 us; error 40 us; skew 500 ppm
Aug 08 07:14:35 dist-test-slave-znh6 krb5kdc[23607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:35.628083 23972 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:14:35.628510 23972 webserver.cc:460] Webserver started at http://127.30.54.66:42413/ using document root <none> and password file <none>
I20260808 07:14:35.628750 23972 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:35.628821 23972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:35.628978 23972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:35.630030 23972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "91021ee7e0d245e8973ccebe791e00c1"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "0a565d9e3982e66e623a8def85e1f183"
server_key_iv: "413e3ba454f39eee6b776c6b5f32982f"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.630399 23972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "91021ee7e0d245e8973ccebe791e00c1"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "0a565d9e3982e66e623a8def85e1f183"
server_key_iv: "413e3ba454f39eee6b776c6b5f32982f"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.631831 23972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:14:35.632652 23997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.632835 23972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:14:35.632938 23972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "91021ee7e0d245e8973ccebe791e00c1"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "0a565d9e3982e66e623a8def85e1f183"
server_key_iv: "413e3ba454f39eee6b776c6b5f32982f"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.633025 23972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:35.653563 23972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:35.655622 23972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:35.655798 23972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:35.656059 23972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:35.656383 23972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:35.656441 23972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.656486 23972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:35.656507 23972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.662693 23972 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:45347
I20260808 07:14:35.662742 24156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:45347 every 8 connection(s)
I20260808 07:14:35.663095 23972 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:14:35 dist-test-slave-znh6 krb5kdc[23607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:35.669821 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 23972
I20260808 07:14:35.669958 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:14:35.670189 30937 external_mini_cluster.cc:1428] Setting key 207c77b413a8cc444810a7c5afcbdba9
I20260808 07:14:35.672662 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.663802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36053 (local address 127.30.54.66:49927)
0808 07:14:35.663981 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:35.664023 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:35.664510 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:35.664726 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:35.664729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:35.665304 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:35.665504 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.665508 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.666159 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.666159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.667070 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.667073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.667417 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.667418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.667544 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.668028 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:35.668036 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:35.669373 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:35.671349 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:35.671356 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:35.671358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:35.671588 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:35.671799 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:35.671800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:35.671800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:35.671902 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:35.672190 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:35.672194 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:35.672300 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 07:14:35.672379 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:14:35.672510 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260808 07:14:35.672659 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.663859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:49927 (local address 127.30.54.126:36053)
0808 07:14:35.663912 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.663915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.664096 (+ 181us) server_negotiation.cc:407] Connection header received
0808 07:14:35.664601 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.664602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.664611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.664672 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:35.665583 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.666077 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.667143 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.667329 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.667386 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.669481 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:35.669483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:35.669483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:35.669491 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:35.671266 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.671662 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.671664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.671664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.671730 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.671969 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.671970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.671971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.672125 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:35.672196 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.672424 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.672513 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:14:35.673454 24157 heartbeater.cc:347] Connected to a master server at 127.30.54.126:36053
I20260808 07:14:35.673564 24157 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:35.673732 24157 heartbeater.cc:511] Master 127.30.54.126:36053 requested a full tablet report, sending...
I20260808 07:14:35.674124 23674 ts_manager.cc:194] Registered new tserver with Master: 91021ee7e0d245e8973ccebe791e00c1 (127.30.54.66:45347)
I20260808 07:14:35.674464 23674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:49927
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:14:35.697028 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36053
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:35.783334 24164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:35.783602 24164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:35.783639 24164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:35.784816 24164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:35.784905 24164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:35.784951 24164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:14:35.786422 24164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:36053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:35.786692 24164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:35.786938 24164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:35.792078 24171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:35.792117 24175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:35.792083 24172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:35.792148 24164 server_base.cc:1034] running on GCE node
I20260808 07:14:35.792449 24164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:35.792747 24164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:35.793910 24164 hybrid_clock.cc:648] HybridClock initialized: now 1786173275793897 us; error 44 us; skew 500 ppm
Aug 08 07:14:35 dist-test-slave-znh6 krb5kdc[23607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:35.796451 24164 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:14:35.796933 24164 webserver.cc:460] Webserver started at http://127.30.54.67:43421/ using document root <none> and password file <none>
I20260808 07:14:35.797175 24164 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:35.797245 24164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:35.797402 24164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:35.798439 24164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "782848b82b28489ab3fd9fc88820ce66"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "1f831a102f218fdefca0566554839b26"
server_key_iv: "e4a8dce5928a4ddf7ffc6513d2d90b7f"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.798864 24164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "782848b82b28489ab3fd9fc88820ce66"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "1f831a102f218fdefca0566554839b26"
server_key_iv: "e4a8dce5928a4ddf7ffc6513d2d90b7f"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.800215 24164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:35.800987 24184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.801226 24164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:35.801328 24164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "782848b82b28489ab3fd9fc88820ce66"
format_stamp: "Formatted at 2026-08-08 07:14:35 on dist-test-slave-znh6"
server_key: "1f831a102f218fdefca0566554839b26"
server_key_iv: "e4a8dce5928a4ddf7ffc6513d2d90b7f"
server_key_version: "encryptionkey@0"
I20260808 07:14:35.801417 24164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:35.826423 24164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:35.828351 24164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:35.828517 24164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:35.828774 24164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:35.829121 24164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:35.829176 24164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.829221 24164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:35.829247 24164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:35.835484 24164 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:34667
I20260808 07:14:35.835539 24350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:34667 every 8 connection(s)
I20260808 07:14:35.835917 24164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:14:35 dist-test-slave-znh6 krb5kdc[23607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173275, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:35.842247 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 24164
I20260808 07:14:35.842370 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:14:35.842582 30937 external_mini_cluster.cc:1428] Setting key 35a9303a050ba5f4d68a7c4f7ea9b10c
I20260808 07:14:35.845733 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.836716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:35201 (local address 127.30.54.126:36053)
0808 07:14:35.836777 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.836779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.837031 (+ 252us) server_negotiation.cc:407] Connection header received
0808 07:14:35.837586 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.837587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.837601 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.837653 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:35.838627 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.839215 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.840218 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.840440 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.840521 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.842701 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:35.842703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:35.842703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:35.842712 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:35.844340 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.844759 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.844760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.844761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.844824 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.845091 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.845092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.845092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.845212 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:35.845285 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.845561 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.845642 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:14:35.845942 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.836706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:36053 (local address 127.30.54.67:35201)
0808 07:14:35.836902 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:35.836936 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:35.837455 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:35.837720 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:35.837724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:35.838319 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:35.838517 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.838522 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.839314 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.839315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.840117 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.840119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.840531 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.840532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.840657 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.841112 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:35.841121 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:35.842586 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:35.844428 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:35.844436 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:35.844438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:35.844665 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:35.844896 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:35.844897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:35.844897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:35.845013 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:35.845287 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:35.845294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:35.845429 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 07:14:35.845536 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:14:35.845693 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":120,"threads_started":1}
I20260808 07:14:35.846386 24352 heartbeater.cc:347] Connected to a master server at 127.30.54.126:36053
I20260808 07:14:35.846493 24352 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:35.846676 24352 heartbeater.cc:511] Master 127.30.54.126:36053 requested a full tablet report, sending...
I20260808 07:14:35.847069 23674 ts_manager.cc:194] Registered new tserver with Master: 782848b82b28489ab3fd9fc88820ce66 (127.30.54.67:34667)
I20260808 07:14:35.847467 23674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:35201
I20260808 07:14:35.854666 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:14:35.865504 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.856473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.30.54.126:36053)
0808 07:14:35.856540 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.856542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.858988 (+ 2446us) server_negotiation.cc:407] Connection header received
0808 07:14:35.859093 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.859093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.859103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.859154 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:35.859813 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.860293 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.861111 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.861321 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.861384 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.862853 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:35.862855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:35.862855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:35.862863 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:35.864296 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.864662 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.864664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.864664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.864726 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:35.864933 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:35.864934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:35.864935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:35.865096 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:35.865207 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.865320 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.865398 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:14:35.867429 23674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:14:35.870363 23674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:14:35.877876 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.873446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:34667 (local address 127.0.0.1:55052)
0808 07:14:35.873567 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:35.873611 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:35.873934 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:35.874252 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:35.874255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:35.874258 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:35.874330 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.874332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.875415 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.875416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.876807 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.876809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.877573 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.877574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.877654 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.877668 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:14:35.877737 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:14:35.877797 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":76,"threads_started":1}
I20260808 07:14:35.878317 24374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.873726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55052 (local address 127.30.54.67:34667)
0808 07:14:35.873926 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.873931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.873945 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:14:35.874053 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.874057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.874075 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.874154 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:35.874411 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.875310 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.876878 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.877476 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.877574 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.877637 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.878198 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.878241 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260808 07:14:35.878399 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.873476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:45347 (local address 127.0.0.1:50666)
0808 07:14:35.873601 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:35.873614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:35.873679 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:35.874308 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:35.874308 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:35.874310 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:35.874348 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.874352 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.875789 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.875790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.877050 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.877055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.878129 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.878131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.878207 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.878212 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:14:35.878248 (+ 36us) client_negotiation.cc:240] Negotiation successful
0808 07:14:35.878302 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":81,"threads_started":1}
I20260808 07:14:35.879177 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.873965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50666 (local address 127.30.54.66:45347)
0808 07:14:35.874142 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.874146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.874155 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:35.874194 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.874196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.874209 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.874424 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:35.874495 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.875652 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.877173 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.878017 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.878917 (+ 900us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.878996 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.879029 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.879082 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260808 07:14:35.879222 24256 tablet_service.cc:1467] Processing CreateTablet for tablet cf788f56fcae46918d48af05cd00eb18 (DEFAULT_TABLE table=test-table [id=2d93f501cf0d42f7ab6c8ac6bc19e6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:35.879491 24375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.873799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37906 (local address 127.30.54.65:40383)
0808 07:14:35.874020 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.874023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.874033 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:14:35.874075 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.874077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.874094 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.874174 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:35.874432 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.875652 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.877798 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.878556 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.878632 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.878695 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.879372 (+ 677us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.879421 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":78,"threads_started":1}
I20260808 07:14:35.879040 24373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.873686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40383 (local address 127.0.0.1:37906)
0808 07:14:35.873814 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:35.873824 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:35.873864 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:35.874260 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:35.874262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:35.874264 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:35.874348 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.874350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.875764 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.875765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.877715 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.877718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.878781 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.878782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.878866 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.878871 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:14:35.878935 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:14:35.878975 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":51,"thread_start_us":93,"threads_started":1}
I20260808 07:14:35.879617 24256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf788f56fcae46918d48af05cd00eb18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:35.879974 24064 tablet_service.cc:1467] Processing CreateTablet for tablet cf788f56fcae46918d48af05cd00eb18 (DEFAULT_TABLE table=test-table [id=2d93f501cf0d42f7ab6c8ac6bc19e6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:35.880312 24064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf788f56fcae46918d48af05cd00eb18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:35.883474 24379 tablet_bootstrap.cc:492] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: Bootstrap starting.
I20260808 07:14:35.884192 24379 tablet_bootstrap.cc:654] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:35.884759 24379 log.cc:824] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:35.885131 24380 tablet_bootstrap.cc:492] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: Bootstrap starting.
I20260808 07:14:35.885630 23867 tablet_service.cc:1467] Processing CreateTablet for tablet cf788f56fcae46918d48af05cd00eb18 (DEFAULT_TABLE table=test-table [id=2d93f501cf0d42f7ab6c8ac6bc19e6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:35.886011 23867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf788f56fcae46918d48af05cd00eb18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:35.886029 24380 tablet_bootstrap.cc:654] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:35.886502 24380 log.cc:824] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:35.887264 24380 tablet_bootstrap.cc:492] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: No bootstrap required, opened a new log
I20260808 07:14:35.887348 24380 ts_tablet_manager.cc:1397] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:14:35.888686 24380 raft_consensus.cc:348] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } }
I20260808 07:14:35.888845 24380 raft_consensus.cc:374] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:35.888895 24380 raft_consensus.cc:729] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91021ee7e0d245e8973ccebe791e00c1, State: Initialized, Role: FOLLOWER
I20260808 07:14:35.889015 24380 consensus_queue.cc:260] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } }
I20260808 07:14:35.889321 24157 heartbeater.cc:503] Master 127.30.54.126:36053 was elected leader, sending a full tablet report...
I20260808 07:14:35.889513 24389 tablet_bootstrap.cc:492] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5: Bootstrap starting.
I20260808 07:14:35.889883 24379 tablet_bootstrap.cc:492] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: No bootstrap required, opened a new log
I20260808 07:14:35.889959 24379 ts_tablet_manager.cc:1397] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 07:14:35.890276 24389 tablet_bootstrap.cc:654] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:35.890784 24389 log.cc:824] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:35.890960 24380 ts_tablet_manager.cc:1428] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 07:14:35.892028 24379 raft_consensus.cc:348] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } }
I20260808 07:14:35.892221 24379 raft_consensus.cc:374] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:35.892297 24379 raft_consensus.cc:729] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782848b82b28489ab3fd9fc88820ce66, State: Initialized, Role: FOLLOWER
I20260808 07:14:35.892418 24379 consensus_queue.cc:260] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } }
I20260808 07:14:35.892711 24379 ts_tablet_manager.cc:1428] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:35.892735 24352 heartbeater.cc:503] Master 127.30.54.126:36053 was elected leader, sending a full tablet report...
I20260808 07:14:35.893378 24389 tablet_bootstrap.cc:492] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5: No bootstrap required, opened a new log
I20260808 07:14:35.893450 24389 ts_tablet_manager.cc:1397] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 07:14:35.895730 24389 raft_consensus.cc:348] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } }
I20260808 07:14:35.895908 24389 raft_consensus.cc:374] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:35.895946 24389 raft_consensus.cc:729] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dee6b3edaa640ec85945813f7a1f0f5, State: Initialized, Role: FOLLOWER
I20260808 07:14:35.896126 24389 consensus_queue.cc:260] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } }
I20260808 07:14:35.896461 24389 ts_tablet_manager.cc:1428] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:35.896519 23966 heartbeater.cc:503] Master 127.30.54.126:36053 was elected leader, sending a full tablet report...
I20260808 07:14:35.910410 24391 raft_consensus.cc:482] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:35.910563 24391 raft_consensus.cc:504] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } }
I20260808 07:14:35.911000 24391 leader_election.cc:288] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dee6b3edaa640ec85945813f7a1f0f5 (127.30.54.65:40383), 91021ee7e0d245e8973ccebe791e00c1 (127.30.54.66:45347)
W20260808 07:14:35.913635 24159 tablet.cc:2369] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:14:35.914474 24375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.911238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:38223 (local address 127.30.54.65:40383)
0808 07:14:35.911284 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.911286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.911459 (+ 173us) server_negotiation.cc:407] Connection header received
0808 07:14:35.911581 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.911582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.911592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.911645 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:35.911848 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.912538 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.913538 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.914092 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.914163 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.914172 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.914368 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.914405 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:14:35.914705 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.911153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40383 (local address 127.30.54.67:38223)
0808 07:14:35.911397 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:35.911407 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:35.911487 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:35.911666 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:35.911667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:35.911669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:35.911762 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.911764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.912600 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.912601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.913396 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.913398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.914205 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.914206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.914276 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.914280 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:35.914570 (+ 290us) client_negotiation.cc:240] Negotiation successful
0808 07:14:35.914616 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":60,"thread_start_us":126,"threads_started":1}
I20260808 07:14:35.914915 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.911153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:45347 (local address 127.30.54.67:59041)
0808 07:14:35.911199 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:35.911214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:35.911269 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:35.911420 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:35.911422 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:35.911423 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:35.911493 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.911495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.912541 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.912542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.913859 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:35.913863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.914628 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.914629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:35.914711 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.914715 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:35.914764 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:14:35.914840 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 07:14:35.915113 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.911259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:59041 (local address 127.30.54.66:45347)
0808 07:14:35.911305 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.911307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.911316 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:35.911347 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.911348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.911356 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.911417 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:35.911692 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.912443 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.913947 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.914519 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.914589 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.914597 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.915011 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.915056 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:14:35.915122 23898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf788f56fcae46918d48af05cd00eb18" candidate_uuid: "782848b82b28489ab3fd9fc88820ce66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" is_pre_election: true
I20260808 07:14:35.915274 23898 raft_consensus.cc:2454] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 782848b82b28489ab3fd9fc88820ce66 in term 0.
I20260808 07:14:35.915447 24093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf788f56fcae46918d48af05cd00eb18" candidate_uuid: "782848b82b28489ab3fd9fc88820ce66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91021ee7e0d245e8973ccebe791e00c1" is_pre_election: true
I20260808 07:14:35.915515 24189 leader_election.cc:302] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 782848b82b28489ab3fd9fc88820ce66, 9dee6b3edaa640ec85945813f7a1f0f5; no voters:
I20260808 07:14:35.915592 24093 raft_consensus.cc:2454] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 782848b82b28489ab3fd9fc88820ce66 in term 0.
I20260808 07:14:35.915652 24391 raft_consensus.cc:2781] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:14:35.915719 24391 raft_consensus.cc:482] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:14:35.915746 24391 raft_consensus.cc:3037] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:35.916450 24391 raft_consensus.cc:504] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } }
I20260808 07:14:35.916580 24391 leader_election.cc:288] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [CANDIDATE]: Term 1 election: Requested vote from peers 9dee6b3edaa640ec85945813f7a1f0f5 (127.30.54.65:40383), 91021ee7e0d245e8973ccebe791e00c1 (127.30.54.66:45347)
I20260808 07:14:35.916726 24093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf788f56fcae46918d48af05cd00eb18" candidate_uuid: "782848b82b28489ab3fd9fc88820ce66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91021ee7e0d245e8973ccebe791e00c1"
I20260808 07:14:35.916750 23898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf788f56fcae46918d48af05cd00eb18" candidate_uuid: "782848b82b28489ab3fd9fc88820ce66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dee6b3edaa640ec85945813f7a1f0f5"
I20260808 07:14:35.916832 24093 raft_consensus.cc:3037] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:35.916826 23898 raft_consensus.cc:3037] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:35.917506 24093 raft_consensus.cc:2454] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 782848b82b28489ab3fd9fc88820ce66 in term 1.
I20260808 07:14:35.917549 23898 raft_consensus.cc:2454] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 782848b82b28489ab3fd9fc88820ce66 in term 1.
I20260808 07:14:35.917716 24190 leader_election.cc:302] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 782848b82b28489ab3fd9fc88820ce66, 91021ee7e0d245e8973ccebe791e00c1; no voters:
I20260808 07:14:35.917822 24391 raft_consensus.cc:2781] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:35.917971 24391 raft_consensus.cc:686] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 1 LEADER]: Becoming Leader. State: Replica: 782848b82b28489ab3fd9fc88820ce66, State: Running, Role: LEADER
I20260808 07:14:35.918071 24391 consensus_queue.cc:237] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } }
I20260808 07:14:35.918884 23674 catalog_manager.cc:5495] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 782848b82b28489ab3fd9fc88820ce66 (127.30.54.67). New cstate: current_term: 1 leader_uuid: "782848b82b28489ab3fd9fc88820ce66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "782848b82b28489ab3fd9fc88820ce66" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 34667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260808 07:14:35.935206 24374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.932947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55058 (local address 127.30.54.67:34667)
0808 07:14:35.933005 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:35.933006 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:35.933015 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:35.933052 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:35.933057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:35.933073 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:35.933131 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:14:35.933269 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.933723 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.934445 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:35.934646 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:35.934710 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:35.934914 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:14:35.935015 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:14:35.935082 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:35.935119 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:14:35.935138 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:14:35.937047 23898 raft_consensus.cc:1261] T cf788f56fcae46918d48af05cd00eb18 P 9dee6b3edaa640ec85945813f7a1f0f5 [term 1 FOLLOWER]: Refusing update from remote peer 782848b82b28489ab3fd9fc88820ce66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:35.937049 24093 raft_consensus.cc:1261] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [term 1 FOLLOWER]: Refusing update from remote peer 782848b82b28489ab3fd9fc88820ce66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:35.937418 24398 consensus_queue.cc:1035] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91021ee7e0d245e8973ccebe791e00c1" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:35.937557 24391 consensus_queue.cc:1035] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dee6b3edaa640ec85945813f7a1f0f5" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:35.939986 24403 mvcc.cc:205] Tried to move back new op lower bound from 7316165738235527168 to 7316165738161033216. Current Snapshot: MvccSnapshot[applied={T|T < 7316165738235527168}]
I20260808 07:14:35.940200 24404 mvcc.cc:205] Tried to move back new op lower bound from 7316165738235527168 to 7316165738161033216. Current Snapshot: MvccSnapshot[applied={T|T < 7316165738235527168}]
I20260808 07:14:35.942133 23674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:35.942209 23674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:35.942912 23674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef8c63498f8a446cb33716093c09e641: Sending DeleteTablet for 3 replicas of tablet cf788f56fcae46918d48af05cd00eb18
I20260808 07:14:35.943277 24064 tablet_service.cc:1514] Processing DeleteTablet for tablet cf788f56fcae46918d48af05cd00eb18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:35 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:50666
I20260808 07:14:35.943320 24256 tablet_service.cc:1514] Processing DeleteTablet for tablet cf788f56fcae46918d48af05cd00eb18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:35 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:55052
I20260808 07:14:35.943320 23867 tablet_service.cc:1514] Processing DeleteTablet for tablet cf788f56fcae46918d48af05cd00eb18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:35 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:37906
I20260808 07:14:35.943602 24418 tablet_replica.cc:330] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: stopping tablet replica
I20260808 07:14:35.943706 24418 raft_consensus.cc:2229] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:14:35.943773 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 23777
I20260808 07:14:35.949000 24418 raft_consensus.cc:2258] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:35.949503 24418 ts_tablet_manager.cc:1905] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:35.950762 24416 tablet_replica.cc:330] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: stopping tablet replica
I20260808 07:14:35.950917 24416 raft_consensus.cc:2229] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:35.951020 24418 ts_tablet_manager.cc:1918] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:35.951043 24416 raft_consensus.cc:2258] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:35.951112 24418 log.cc:1196] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/cf788f56fcae46918d48af05cd00eb18
I20260808 07:14:35.952037 24418 ts_tablet_manager.cc:1939] T cf788f56fcae46918d48af05cd00eb18 P 782848b82b28489ab3fd9fc88820ce66: Deleting consensus metadata
I20260808 07:14:35.952437 23655 catalog_manager.cc:4841] TS 782848b82b28489ab3fd9fc88820ce66 (127.30.54.67:34667): tablet cf788f56fcae46918d48af05cd00eb18 (table test-table [id=2d93f501cf0d42f7ab6c8ac6bc19e6c8]) successfully deleted
I20260808 07:14:35.953370 24416 ts_tablet_manager.cc:1905] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:35.955024 24416 ts_tablet_manager.cc:1918] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:35.955109 24416 log.cc:1196] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/cf788f56fcae46918d48af05cd00eb18
I20260808 07:14:35.955441 24416 ts_tablet_manager.cc:1939] T cf788f56fcae46918d48af05cd00eb18 P 91021ee7e0d245e8973ccebe791e00c1: Deleting consensus metadata
I20260808 07:14:35.955873 23656 catalog_manager.cc:4841] TS 91021ee7e0d245e8973ccebe791e00c1 (127.30.54.66:45347): tablet cf788f56fcae46918d48af05cd00eb18 (table test-table [id=2d93f501cf0d42f7ab6c8ac6bc19e6c8]) successfully deleted
W20260808 07:14:35.958626 23655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:40383 (error 108)
I20260808 07:14:35.958884 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 23972
I20260808 07:14:35.959157 24373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:35.959041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:40383 (local address 127.0.0.1:37912)
0808 07:14:35.959086 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:35.959119 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:40383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 07:14:35.961995 23655 catalog_manager.cc:4584] TS 9dee6b3edaa640ec85945813f7a1f0f5 (127.30.54.65:40383): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf788f56fcae46918d48af05cd00eb18: Network error: Client connection negotiation failed: client connection to 127.30.54.65:40383: connect: Connection refused (error 111)
I20260808 07:14:35.966202 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 24164
I20260808 07:14:35.980059 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 23629
2026-08-08T07:14:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2123 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T07:14:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:36Z Disabled control of system clock
I20260808 07:14:36.011268 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41509
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:41509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:14:36.093649 24429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:36.093856 24429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:36.093878 24429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:36.095116 24429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:36.095160 24429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:36.095172 24429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:36.095185 24429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:36.095197 24429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:36.096553 24429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:41509
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:36.096796 24429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:36.097082 24429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:36.102035 24434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:36.102008 24437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:36.102010 24435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:36.102059 24429 server_base.cc:1034] running on GCE node
I20260808 07:14:36.102488 24429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:36.102782 24429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:36.103987 24429 hybrid_clock.cc:648] HybridClock initialized: now 1786173276103963 us; error 43 us; skew 500 ppm
I20260808 07:14:36.105355 24429 webserver.cc:460] Webserver started at http://127.30.54.126:46473/ using document root <none> and password file <none>
I20260808 07:14:36.105599 24429 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:36.105670 24429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:36.105844 24429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:36.106985 24429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "54726bd9cace40bba735d4171aa265a8"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "0d8b8f67ef2e9a5cd04bdb8b095c846f"
server_key_iv: "4c300724ef7ec3a09bf7f8a3a7d255a9"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.107349 24429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "54726bd9cace40bba735d4171aa265a8"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "0d8b8f67ef2e9a5cd04bdb8b095c846f"
server_key_iv: "4c300724ef7ec3a09bf7f8a3a7d255a9"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.108691 24429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:14:36.109477 24447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.109679 24429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:36.109788 24429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "54726bd9cace40bba735d4171aa265a8"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "0d8b8f67ef2e9a5cd04bdb8b095c846f"
server_key_iv: "4c300724ef7ec3a09bf7f8a3a7d255a9"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.109871 24429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:36.120378 24429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:36.120666 24429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:36.120826 24429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:36.125399 24429 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:41509
I20260808 07:14:36.125435 24524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:41509 every 8 connection(s)
I20260808 07:14:36.125793 24429 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:36.126789 24525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:36.129138 24525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8: Bootstrap starting.
I20260808 07:14:36.129753 24525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:36.130064 24525 log.cc:824] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:36.130771 24525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8: No bootstrap required, opened a new log
I20260808 07:14:36.132406 24525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54726bd9cace40bba735d4171aa265a8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41509 } }
I20260808 07:14:36.132555 24525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:36.132578 24525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54726bd9cace40bba735d4171aa265a8, State: Initialized, Role: FOLLOWER
I20260808 07:14:36.132753 24525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54726bd9cace40bba735d4171aa265a8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41509 } }
I20260808 07:14:36.132836 24525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:36.132859 24525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:36.132920 24525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:36.133452 24525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54726bd9cace40bba735d4171aa265a8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41509 } }
I20260808 07:14:36.133571 24525 leader_election.cc:302] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54726bd9cace40bba735d4171aa265a8; no voters:
I20260808 07:14:36.133764 24525 leader_election.cc:288] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:36.133812 24530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:36.134027 24530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [term 1 LEADER]: Becoming Leader. State: Replica: 54726bd9cace40bba735d4171aa265a8, State: Running, Role: LEADER
I20260808 07:14:36.134053 24525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:36.134220 24530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54726bd9cace40bba735d4171aa265a8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41509 } }
I20260808 07:14:36.134655 24534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54726bd9cace40bba735d4171aa265a8. Latest consensus state: current_term: 1 leader_uuid: "54726bd9cace40bba735d4171aa265a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54726bd9cace40bba735d4171aa265a8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41509 } } }
I20260808 07:14:36.134745 24533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54726bd9cace40bba735d4171aa265a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54726bd9cace40bba735d4171aa265a8" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41509 } } }
I20260808 07:14:36.134794 24534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:36.134833 24533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:36.135049 24544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:36.135772 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 24429
I20260808 07:14:36.135890 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:36.136094 30937 external_mini_cluster.cc:1428] Setting key 27a1a54dc504b076fa61f1a12376ae45
I20260808 07:14:36.136405 24544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:36.138145 24544 catalog_manager.cc:1347] Generated new cluster ID: bf2827711a41422485e2048b1ee5ccca
I20260808 07:14:36.138201 24544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:36.144178 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.136603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.30.54.126:41509)
0808 07:14:36.136955 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.136964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.137122 (+ 158us) server_negotiation.cc:407] Connection header received
0808 07:14:36.137281 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.137288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.137359 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.137494 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:36.137842 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.139276 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.140459 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.141095 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.141555 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.143075 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:36.143079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:36.143079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:36.143312 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:36.143356 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:36.143371 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:36.143702 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.143737 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.143829 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":97,"threads_started":1}
I20260808 07:14:36.159427 24544 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:36.159950 24544 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:36.172032 24544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8: Generated new TSK 0
I20260808 07:14:36.172235 24544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:36.179288 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41509
--builtin_ntp_servers=127.30.54.84:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:14:36.262383 24563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:36.262599 24563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:36.262622 24563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:36.263844 24563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:36.263891 24563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:36.263917 24563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:14:36.265362 24563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:36.265632 24563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:36.265941 24563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:36.271095 24572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:36.271104 24577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:36.271106 24569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:36.271368 24563 server_base.cc:1034] running on GCE node
I20260808 07:14:36.271591 24563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:36.271839 24563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:36.273018 24563 hybrid_clock.cc:648] HybridClock initialized: now 1786173276273003 us; error 53 us; skew 500 ppm
I20260808 07:14:36.274252 24563 webserver.cc:460] Webserver started at http://127.30.54.65:46509/ using document root <none> and password file <none>
I20260808 07:14:36.274514 24563 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:36.274591 24563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:36.274744 24563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:36.275830 24563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "81e51ba851c7462fa4dbf0edbb6af4de"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "ea8a65a0a055b9fb938d2c78b904f1e3"
server_key_iv: "13dda948e1be808f56fba89da398acb7"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.276206 24563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "81e51ba851c7462fa4dbf0edbb6af4de"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "ea8a65a0a055b9fb938d2c78b904f1e3"
server_key_iv: "13dda948e1be808f56fba89da398acb7"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.277552 24563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:14:36.278280 24583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.278458 24563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:36.278563 24563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "81e51ba851c7462fa4dbf0edbb6af4de"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "ea8a65a0a055b9fb938d2c78b904f1e3"
server_key_iv: "13dda948e1be808f56fba89da398acb7"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.278643 24563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:36.293645 24563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:36.293972 24563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:36.294131 24563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:36.294374 24563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:36.294724 24563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:36.294790 24563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.294845 24563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:36.294878 24563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.300829 24563 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:39683
I20260808 07:14:36.300925 24750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:39683 every 8 connection(s)
I20260808 07:14:36.301240 24563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:36.305197 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 24563
I20260808 07:14:36.305282 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:14:36.305514 30937 external_mini_cluster.cc:1428] Setting key c0a04f8a8a7f93d1b9a70652932edbc9
I20260808 07:14:36.306684 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.302030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:33315 (local address 127.30.54.126:41509)
0808 07:14:36.302104 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.302106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.302354 (+ 248us) server_negotiation.cc:407] Connection header received
0808 07:14:36.302880 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.302881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.302894 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.302952 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:36.303628 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.304128 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.305134 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.305382 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.305432 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.306423 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:36.306425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:36.306425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:36.306430 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:36.306444 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:36.306448 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:36.306503 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.306575 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.306608 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:14:36.306982 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.301997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41509 (local address 127.30.54.65:33315)
0808 07:14:36.302222 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:36.302262 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:36.302742 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:36.303023 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:36.303027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:36.303035 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:36.303555 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.303559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.304214 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.304214 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.305046 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.305050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.305452 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.305454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.305621 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.306246 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:14:36.306246 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:36.306284 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:14:36.306310 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:36.306496 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:36.306502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:36.306504 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:14:36.306577 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:14:36.306677 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":122,"threads_started":1}
I20260808 07:14:36.307060 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41509
--builtin_ntp_servers=127.30.54.84:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:14:36.307433 24751 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41509
I20260808 07:14:36.307523 24751 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:36.307721 24751 heartbeater.cc:511] Master 127.30.54.126:41509 requested a full tablet report, sending...
I20260808 07:14:36.308228 24472 ts_manager.cc:194] Registered new tserver with Master: 81e51ba851c7462fa4dbf0edbb6af4de (127.30.54.65:39683)
I20260808 07:14:36.308930 24472 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.65:33315
W20260808 07:14:36.390973 24755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:36.391170 24755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:36.391192 24755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:36.392283 24755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:36.392318 24755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:36.392342 24755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:14:36.393604 24755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:36.393790 24755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:36.394035 24755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:36.399114 24763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:36.399204 24755 server_base.cc:1034] running on GCE node
W20260808 07:14:36.399353 24765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:36.399397 24762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:36.399628 24755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:36.399966 24755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:36.401121 24755 hybrid_clock.cc:648] HybridClock initialized: now 1786173276401077 us; error 59 us; skew 500 ppm
I20260808 07:14:36.402730 24755 webserver.cc:460] Webserver started at http://127.30.54.66:44031/ using document root <none> and password file <none>
I20260808 07:14:36.403075 24755 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:36.403165 24755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:36.403318 24755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:36.404714 24755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "cda8411512274a26a35f7401ffb50e3c"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "3dc7af81c4fba360b5556d7615dc4b1b"
server_key_iv: "1659a7e99902d35a2b8ed27cd2aa7547"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.405141 24755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "cda8411512274a26a35f7401ffb50e3c"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "3dc7af81c4fba360b5556d7615dc4b1b"
server_key_iv: "1659a7e99902d35a2b8ed27cd2aa7547"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.406637 24755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 07:14:36.407418 24773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.407577 24755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.407671 24755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "cda8411512274a26a35f7401ffb50e3c"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "3dc7af81c4fba360b5556d7615dc4b1b"
server_key_iv: "1659a7e99902d35a2b8ed27cd2aa7547"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.407727 24755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:36.421797 24755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:36.422057 24755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:36.422161 24755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:36.422353 24755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:36.422746 24755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:36.422791 24755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.422864 24755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:36.422894 24755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.428638 24755 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:45619
I20260808 07:14:36.428723 24933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:45619 every 8 connection(s)
I20260808 07:14:36.429056 24755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:14:36.432849 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 24755
I20260808 07:14:36.432976 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:14:36.433195 30937 external_mini_cluster.cc:1428] Setting key 17ed85abeed1894a9f7f475c3ff66131
I20260808 07:14:36.434556 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41509
--builtin_ntp_servers=127.30.54.84:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:14:36.434592 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.429847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:35785 (local address 127.30.54.126:41509)
0808 07:14:36.429931 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.429934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.430146 (+ 212us) server_negotiation.cc:407] Connection header received
0808 07:14:36.430644 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.430645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.430657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.430718 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:36.431453 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.431955 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.432992 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.433302 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.433385 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.434217 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:36.434218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:36.434218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:36.434224 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:36.434242 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:36.434247 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:36.434311 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.434448 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.434478 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:14:36.434974 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.429790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41509 (local address 127.30.54.66:35785)
0808 07:14:36.430020 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:36.430053 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:36.430518 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:36.430803 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:36.430806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:36.430858 (+ 52us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:36.431370 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.431375 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.432049 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.432050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.432862 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.432864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.433410 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.433411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.433567 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.434058 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:14:36.434058 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:36.434096 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:14:36.434124 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:36.434328 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:36.434362 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:36.434367 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:14:36.434437 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:14:36.434516 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":122,"threads_started":1}
I20260808 07:14:36.435416 24934 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41509
I20260808 07:14:36.435501 24934 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:36.435689 24934 heartbeater.cc:511] Master 127.30.54.126:41509 requested a full tablet report, sending...
I20260808 07:14:36.436051 24472 ts_manager.cc:194] Registered new tserver with Master: cda8411512274a26a35f7401ffb50e3c (127.30.54.66:45619)
I20260808 07:14:36.436571 24472 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.66:35785
W20260808 07:14:36.516624 24937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:36.516813 24937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:36.516834 24937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:36.517961 24937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:36.518002 24937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:36.518025 24937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:14:36.519369 24937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:41509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:36.519615 24937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:36.519932 24937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:36.525123 24947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:36.525158 24937 server_base.cc:1034] running on GCE node
W20260808 07:14:36.525132 24950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:36.525120 24946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:36.525555 24937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:36.525761 24937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:36.526919 24937 hybrid_clock.cc:648] HybridClock initialized: now 1786173276526903 us; error 27 us; skew 500 ppm
I20260808 07:14:36.528147 24937 webserver.cc:460] Webserver started at http://127.30.54.67:33845/ using document root <none> and password file <none>
I20260808 07:14:36.528348 24937 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:36.528388 24937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:36.528482 24937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:36.529445 24937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "ff4e6abaa5284aa8ae62699da07134ca"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "7b1e4372537c31f43478df4d548d992e"
server_key_iv: "22595065e883029cb0b8ebd59060659d"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.529735 24937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "ff4e6abaa5284aa8ae62699da07134ca"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "7b1e4372537c31f43478df4d548d992e"
server_key_iv: "22595065e883029cb0b8ebd59060659d"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.531061 24937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:14:36.531775 24959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.531956 24937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:36.532056 24937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "ff4e6abaa5284aa8ae62699da07134ca"
format_stamp: "Formatted at 2026-08-08 07:14:36 on dist-test-slave-znh6"
server_key: "7b1e4372537c31f43478df4d548d992e"
server_key_iv: "22595065e883029cb0b8ebd59060659d"
server_key_version: "encryptionkey@0"
I20260808 07:14:36.532128 24937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:36.539783 24937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:36.540045 24937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:36.540184 24937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:36.540400 24937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:36.540702 24937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:36.540755 24937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.540794 24937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:36.540820 24937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:36.546738 24937 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:44521
I20260808 07:14:36.546787 25122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:44521 every 8 connection(s)
I20260808 07:14:36.547189 24937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:14:36.550256 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 24937
I20260808 07:14:36.550359 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:14:36.550570 30937 external_mini_cluster.cc:1428] Setting key 5134695879561bde1e52f5677ea7b304
I20260808 07:14:36.552860 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.547936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:42295 (local address 127.30.54.126:41509)
0808 07:14:36.547985 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.547987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.548215 (+ 228us) server_negotiation.cc:407] Connection header received
0808 07:14:36.548776 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.548777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.548791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.548843 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:36.549544 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.550139 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.551563 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.551843 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.551901 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.552642 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:36.552643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:36.552644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:36.552648 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:36.552661 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:36.552666 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:36.552710 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.552776 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.552801 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:14:36.553112 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.547892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:41509 (local address 127.30.54.67:42295)
0808 07:14:36.548078 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:36.548111 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:36.548650 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:36.548908 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:36.548912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:36.548922 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:36.549435 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.549440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.550260 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.550261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.551476 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.551479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.551923 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.551924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.552033 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.552512 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:14:36.552512 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:36.552540 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:14:36.552563 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:36.552722 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:36.552726 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:36.552728 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:14:36.552780 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:14:36.552859 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260808 07:14:36.553557 25123 heartbeater.cc:347] Connected to a master server at 127.30.54.126:41509
I20260808 07:14:36.553637 25123 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:36.553831 25123 heartbeater.cc:511] Master 127.30.54.126:41509 requested a full tablet report, sending...
I20260808 07:14:36.554167 24472 ts_manager.cc:194] Registered new tserver with Master: ff4e6abaa5284aa8ae62699da07134ca (127.30.54.67:44521)
I20260808 07:14:36.554538 24472 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.67:42295
I20260808 07:14:36.562541 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:14:36.567046 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.564118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54940 (local address 127.30.54.126:41509)
0808 07:14:36.564204 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.564206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.564304 (+ 98us) server_negotiation.cc:407] Connection header received
0808 07:14:36.564379 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.564379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.564391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.564436 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:36.564623 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.565098 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.565871 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.566076 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.566165 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.566779 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:36.566781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:36.566781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:36.566786 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:36.566799 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:36.566803 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:36.566880 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.566940 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.566962 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 07:14:36.568445 24472 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:14:36.569926 24472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:14:36.576010 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.572207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45094 (local address 127.30.54.67:44521)
0808 07:14:36.572502 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.572508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.572519 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:14:36.572553 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.572555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.572568 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.572668 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:36.572853 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.573662 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.574917 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.575618 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.575704 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.575793 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.575889 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.575924 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":109,"threads_started":1}
I20260808 07:14:36.575999 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.572094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:44521 (local address 127.0.0.1:45094)
0808 07:14:36.572196 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:36.572235 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:36.572294 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:36.572694 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:36.572697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:36.572698 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:36.572765 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.572767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.573748 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.573749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.574792 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.574794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.575704 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.575705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.575783 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.575796 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:14:36.575868 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:14:36.575933 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":58,"threads_started":1}
I20260808 07:14:36.576622 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.572298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:45619 (local address 127.0.0.1:60144)
0808 07:14:36.572756 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:36.572766 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:36.572813 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:36.573092 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:36.573093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:36.573095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:36.573133 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.573134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.574359 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.574361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.575312 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.575314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.576354 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.576355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.576421 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.576425 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:36.576503 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:14:36.576548 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260808 07:14:36.576658 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.572422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60144 (local address 127.30.54.66:45619)
0808 07:14:36.572618 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.572622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.572881 (+ 259us) server_negotiation.cc:407] Connection header received
0808 07:14:36.572933 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.572936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.572951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.573022 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:36.573228 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.574296 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.575410 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.576134 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.576213 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.576313 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.576537 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.576579 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":95,"threads_started":1}
I20260808 07:14:36.576856 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.572098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:39683 (local address 127.0.0.1:43282)
0808 07:14:36.572423 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:36.572433 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:36.572477 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:36.572946 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:36.572947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:36.572949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:36.572994 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.572995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.574279 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.574280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.575387 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.575389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.576638 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.576639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.576706 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.576709 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:36.576765 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:14:36.576801 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":93,"threads_started":1}
I20260808 07:14:36.576905 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.572415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.30.54.65:39683)
0808 07:14:36.572711 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.572715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.572726 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:14:36.572767 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.572770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.572786 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.572868 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:36.573057 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.574186 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.575475 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.576172 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.576498 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.576601 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.576783 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.576821 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":96,"threads_started":1}
I20260808 07:14:36.576941 25026 tablet_service.cc:1467] Processing CreateTablet for tablet c27a14bf2af5451e88f2f3b540c961ef (DEFAULT_TABLE table=test-table [id=48c7e27557ce4b9b852966f7249e5b6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:36.577342 25026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c27a14bf2af5451e88f2f3b540c961ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:36.577554 24649 tablet_service.cc:1467] Processing CreateTablet for tablet c27a14bf2af5451e88f2f3b540c961ef (DEFAULT_TABLE table=test-table [id=48c7e27557ce4b9b852966f7249e5b6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:36.577818 24649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c27a14bf2af5451e88f2f3b540c961ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:36.579535 24843 tablet_service.cc:1467] Processing CreateTablet for tablet c27a14bf2af5451e88f2f3b540c961ef (DEFAULT_TABLE table=test-table [id=48c7e27557ce4b9b852966f7249e5b6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:36.579879 24843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c27a14bf2af5451e88f2f3b540c961ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:36.580668 25146 tablet_bootstrap.cc:492] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de: Bootstrap starting.
I20260808 07:14:36.580570 25145 tablet_bootstrap.cc:492] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: Bootstrap starting.
I20260808 07:14:36.581322 25145 tablet_bootstrap.cc:654] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:36.581359 25146 tablet_bootstrap.cc:654] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:36.581665 25145 log.cc:824] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:36.581692 25146 log.cc:824] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:36.582471 25146 tablet_bootstrap.cc:492] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de: No bootstrap required, opened a new log
I20260808 07:14:36.582543 25146 ts_tablet_manager.cc:1397] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:14:36.583173 25145 tablet_bootstrap.cc:492] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: No bootstrap required, opened a new log
I20260808 07:14:36.583576 25145 ts_tablet_manager.cc:1397] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:14:36.584609 25146 raft_consensus.cc:348] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } }
I20260808 07:14:36.584738 25146 raft_consensus.cc:374] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:36.584759 25146 raft_consensus.cc:729] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e51ba851c7462fa4dbf0edbb6af4de, State: Initialized, Role: FOLLOWER
I20260808 07:14:36.584880 25146 consensus_queue.cc:260] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } }
I20260808 07:14:36.585160 25146 ts_tablet_manager.cc:1428] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:36.585223 24751 heartbeater.cc:503] Master 127.30.54.126:41509 was elected leader, sending a full tablet report...
I20260808 07:14:36.585369 25153 tablet_bootstrap.cc:492] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: Bootstrap starting.
I20260808 07:14:36.585837 25145 raft_consensus.cc:348] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } }
I20260808 07:14:36.586005 25153 tablet_bootstrap.cc:654] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:36.586016 25145 raft_consensus.cc:374] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:36.586061 25145 raft_consensus.cc:729] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff4e6abaa5284aa8ae62699da07134ca, State: Initialized, Role: FOLLOWER
I20260808 07:14:36.586174 25145 consensus_queue.cc:260] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } }
I20260808 07:14:36.586323 25153 log.cc:824] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:36.586431 25145 ts_tablet_manager.cc:1428] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:36.586511 25123 heartbeater.cc:503] Master 127.30.54.126:41509 was elected leader, sending a full tablet report...
I20260808 07:14:36.587072 25153 tablet_bootstrap.cc:492] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: No bootstrap required, opened a new log
I20260808 07:14:36.587142 25153 ts_tablet_manager.cc:1397] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:14:36.589061 25153 raft_consensus.cc:348] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } }
I20260808 07:14:36.589177 25153 raft_consensus.cc:374] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:36.589200 25153 raft_consensus.cc:729] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cda8411512274a26a35f7401ffb50e3c, State: Initialized, Role: FOLLOWER
I20260808 07:14:36.589334 25153 consensus_queue.cc:260] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } }
I20260808 07:14:36.589581 25153 ts_tablet_manager.cc:1428] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 07:14:36.589628 24934 heartbeater.cc:503] Master 127.30.54.126:41509 was elected leader, sending a full tablet report...
I20260808 07:14:36.595198 25158 raft_consensus.cc:482] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:36.595270 25158 raft_consensus.cc:504] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } }
I20260808 07:14:36.595634 25158 leader_election.cc:288] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff4e6abaa5284aa8ae62699da07134ca (127.30.54.67:44521), cda8411512274a26a35f7401ffb50e3c (127.30.54.66:45619)
I20260808 07:14:36.599417 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.595792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:44521 (local address 127.30.54.65:60939)
0808 07:14:36.595912 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:36.595926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:36.595985 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:36.596165 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:36.596166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:36.596167 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:36.596235 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.596236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.597118 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.597118 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.598403 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.598407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.599143 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.599144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.599223 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.599227 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:36.599296 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:14:36.599343 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":55}
I20260808 07:14:36.599426 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.595848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:60939 (local address 127.30.54.67:44521)
0808 07:14:36.595890 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.595892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.595969 (+ 77us) server_negotiation.cc:407] Connection header received
0808 07:14:36.596061 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.596062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.596071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.596152 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:36.596304 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.597045 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.598498 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.599057 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.599134 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.599141 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.599306 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.599346 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:14:36.599772 25059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c27a14bf2af5451e88f2f3b540c961ef" candidate_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff4e6abaa5284aa8ae62699da07134ca" is_pre_election: true
I20260808 07:14:36.599814 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.595971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:59201 (local address 127.30.54.66:45619)
0808 07:14:36.596013 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.596015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.596119 (+ 104us) server_negotiation.cc:407] Connection header received
0808 07:14:36.596179 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.596180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.596189 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.596251 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:36.596357 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.597029 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.598498 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.599057 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.599134 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.599141 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.599376 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.599413 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:14:36.599871 25059 raft_consensus.cc:2454] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81e51ba851c7462fa4dbf0edbb6af4de in term 0.
I20260808 07:14:36.600008 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.595898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:45619 (local address 127.30.54.65:59201)
0808 07:14:36.596052 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:36.596061 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:36.596115 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:36.596250 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:36.596251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:36.596253 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:36.596309 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.596310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.597118 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.597118 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.598404 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:36.598407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.599145 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.599146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:36.599245 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.599248 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:36.599298 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:14:36.599339 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":44,"thread_start_us":103,"threads_started":1}
I20260808 07:14:36.600119 24585 leader_election.cc:302] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81e51ba851c7462fa4dbf0edbb6af4de, ff4e6abaa5284aa8ae62699da07134ca; no voters:
I20260808 07:14:36.600279 25158 raft_consensus.cc:2781] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:14:36.600346 25158 raft_consensus.cc:482] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:14:36.600382 25158 raft_consensus.cc:3037] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:36.600466 24865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c27a14bf2af5451e88f2f3b540c961ef" candidate_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cda8411512274a26a35f7401ffb50e3c" is_pre_election: true
I20260808 07:14:36.600569 24865 raft_consensus.cc:2454] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81e51ba851c7462fa4dbf0edbb6af4de in term 0.
I20260808 07:14:36.601174 25158 raft_consensus.cc:504] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } }
I20260808 07:14:36.601310 25158 leader_election.cc:288] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [CANDIDATE]: Term 1 election: Requested vote from peers ff4e6abaa5284aa8ae62699da07134ca (127.30.54.67:44521), cda8411512274a26a35f7401ffb50e3c (127.30.54.66:45619)
I20260808 07:14:36.601490 24865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c27a14bf2af5451e88f2f3b540c961ef" candidate_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cda8411512274a26a35f7401ffb50e3c"
I20260808 07:14:36.601500 25059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c27a14bf2af5451e88f2f3b540c961ef" candidate_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff4e6abaa5284aa8ae62699da07134ca"
I20260808 07:14:36.601567 24865 raft_consensus.cc:3037] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:36.601567 25059 raft_consensus.cc:3037] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:36.602392 25059 raft_consensus.cc:2454] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81e51ba851c7462fa4dbf0edbb6af4de in term 1.
I20260808 07:14:36.602392 24865 raft_consensus.cc:2454] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81e51ba851c7462fa4dbf0edbb6af4de in term 1.
I20260808 07:14:36.602594 24585 leader_election.cc:302] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81e51ba851c7462fa4dbf0edbb6af4de, cda8411512274a26a35f7401ffb50e3c; no voters:
I20260808 07:14:36.602690 25158 raft_consensus.cc:2781] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:36.602905 25158 raft_consensus.cc:686] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [term 1 LEADER]: Becoming Leader. State: Replica: 81e51ba851c7462fa4dbf0edbb6af4de, State: Running, Role: LEADER
I20260808 07:14:36.603020 25158 consensus_queue.cc:237] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } }
I20260808 07:14:36.603747 24472 catalog_manager.cc:5495] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de reported cstate change: term changed from 0 to 1, leader changed from <none> to 81e51ba851c7462fa4dbf0edbb6af4de (127.30.54.65). New cstate: current_term: 1 leader_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e51ba851c7462fa4dbf0edbb6af4de" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 39683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 } health_report { overall_health: UNKNOWN } } }
I20260808 07:14:36.611138 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.608803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43292 (local address 127.30.54.65:39683)
0808 07:14:36.608847 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:36.608849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:36.608858 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:36.608928 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:36.608929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:36.608946 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:36.609009 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:14:36.609104 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.609554 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.610309 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:36.610527 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:36.610602 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:36.610765 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:14:36.610915 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:14:36.611007 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:36.611029 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 07:14:36.611049 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:14:36.612653 25059 raft_consensus.cc:1261] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [term 1 FOLLOWER]: Refusing update from remote peer 81e51ba851c7462fa4dbf0edbb6af4de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:36.612620 24865 raft_consensus.cc:1261] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [term 1 FOLLOWER]: Refusing update from remote peer 81e51ba851c7462fa4dbf0edbb6af4de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:36.612967 25158 consensus_queue.cc:1035] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff4e6abaa5284aa8ae62699da07134ca" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 44521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:36.613101 25163 consensus_queue.cc:1035] T c27a14bf2af5451e88f2f3b540c961ef P 81e51ba851c7462fa4dbf0edbb6af4de [LEADER]: Connected to new peer: Peer: permanent_uuid: "cda8411512274a26a35f7401ffb50e3c" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 45619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:36.616907 25169 mvcc.cc:205] Tried to move back new op lower bound from 7316165741003104256 to 7316165740966473728. Current Snapshot: MvccSnapshot[applied={T|T < 7316165741003104256 or (T in {7316165741003104256})}]
I20260808 07:14:36.617653 24472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:36.617743 24472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:36.618456 24472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54726bd9cace40bba735d4171aa265a8: Sending DeleteTablet for 3 replicas of tablet c27a14bf2af5451e88f2f3b540c961ef
I20260808 07:14:36.618785 24649 tablet_service.cc:1514] Processing DeleteTablet for tablet c27a14bf2af5451e88f2f3b540c961ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:36 UTC) from {username='slave'} at 127.0.0.1:43282
I20260808 07:14:36.618886 25026 tablet_service.cc:1514] Processing DeleteTablet for tablet c27a14bf2af5451e88f2f3b540c961ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:36 UTC) from {username='slave'} at 127.0.0.1:45094
I20260808 07:14:36.618918 24843 tablet_service.cc:1514] Processing DeleteTablet for tablet c27a14bf2af5451e88f2f3b540c961ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:36 UTC) from {username='slave'} at 127.0.0.1:60144
I20260808 07:14:36.619187 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 24563
I20260808 07:14:36.621475 25180 tablet_replica.cc:330] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: stopping tablet replica
I20260808 07:14:36.621575 25180 raft_consensus.cc:2229] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:36.622900 25180 raft_consensus.cc:2258] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:36.624473 25180 ts_tablet_manager.cc:1905] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:36.625001 25183 tablet_replica.cc:330] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: stopping tablet replica
I20260808 07:14:36.625102 25183 raft_consensus.cc:2229] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:36.625193 25183 raft_consensus.cc:2258] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:36.626144 25180 ts_tablet_manager.cc:1918] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:36.626212 25180 log.cc:1196] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/c27a14bf2af5451e88f2f3b540c961ef
I20260808 07:14:36.626442 25180 ts_tablet_manager.cc:1939] T c27a14bf2af5451e88f2f3b540c961ef P ff4e6abaa5284aa8ae62699da07134ca: Deleting consensus metadata
I20260808 07:14:36.626575 25183 ts_tablet_manager.cc:1905] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:36.626874 24450 catalog_manager.cc:4841] TS ff4e6abaa5284aa8ae62699da07134ca (127.30.54.67:44521): tablet c27a14bf2af5451e88f2f3b540c961ef (table test-table [id=48c7e27557ce4b9b852966f7249e5b6b]) successfully deleted
I20260808 07:14:36.631191 25183 ts_tablet_manager.cc:1918] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:36.631254 25183 log.cc:1196] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/c27a14bf2af5451e88f2f3b540c961ef
I20260808 07:14:36.631492 25183 ts_tablet_manager.cc:1939] T c27a14bf2af5451e88f2f3b540c961ef P cda8411512274a26a35f7401ffb50e3c: Deleting consensus metadata
I20260808 07:14:36.631863 24450 catalog_manager.cc:4841] TS cda8411512274a26a35f7401ffb50e3c (127.30.54.66:45619): tablet c27a14bf2af5451e88f2f3b540c961ef (table test-table [id=48c7e27557ce4b9b852966f7249e5b6b]) successfully deleted
W20260808 07:14:36.634784 24451 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:39683 (error 108)
I20260808 07:14:36.635106 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 24755
I20260808 07:14:36.635728 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:36.635242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:39683 (local address 127.0.0.1:43306)
0808 07:14:36.635631 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:36.635686 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365}
W20260808 07:14:36.640674 24451 catalog_manager.cc:4584] TS 81e51ba851c7462fa4dbf0edbb6af4de (127.30.54.65:39683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c27a14bf2af5451e88f2f3b540c961ef: Network error: Client connection negotiation failed: client connection to 127.30.54.65:39683: connect: Connection refused (error 111)
I20260808 07:14:36.646958 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 24937
I20260808 07:14:36.655877 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 24429
2026-08-08T07:14:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (677 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:36 dist-test-slave-znh6 krb5kdc[25189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:36 dist-test-slave-znh6 krb5kdc[25189](info): set up 2 sockets
Aug 08 07:14:36 dist-test-slave-znh6 krb5kdc[25189](info): commencing operation
krb5kdc: starting...
W20260808 07:14:37.888062 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:37 dist-test-slave-znh6 krb5kdc[25189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:14:38.007465 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:46791
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:46791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:38.089421 25206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:38.089643 25206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:38.089680 25206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:38.090804 25206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:38.090896 25206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:38.090929 25206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:38.090963 25206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:38.090987 25206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:38.092446 25206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:46791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:46791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:38.092713 25206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:38.092945 25206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:38.097754 25213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:38.097779 25214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:38.097941 25206 server_base.cc:1034] running on GCE node
W20260808 07:14:38.097781 25217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:38.098263 25206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:38.098569 25206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:38.099736 25206 hybrid_clock.cc:648] HybridClock initialized: now 1786173278099702 us; error 51 us; skew 500 ppm
Aug 08 07:14:38 dist-test-slave-znh6 krb5kdc[25189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173278, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:38.102165 25206 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:14:38.102528 25206 webserver.cc:460] Webserver started at http://127.30.54.126:42975/ using document root <none> and password file <none>
I20260808 07:14:38.102716 25206 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:38.102756 25206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:38.102916 25206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:38.103893 25206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "c8fa640fa7d1406aabbeebb134aeb957"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "9817b607b2f74e2ff4fa4ee9215b4545"
server_key_iv: "dd28250a187a02fb2acbfb13b3a9dfe5"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.104179 25206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "c8fa640fa7d1406aabbeebb134aeb957"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "9817b607b2f74e2ff4fa4ee9215b4545"
server_key_iv: "dd28250a187a02fb2acbfb13b3a9dfe5"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.105400 25206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:14:38.106025 25227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.106194 25206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:38.106273 25206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "c8fa640fa7d1406aabbeebb134aeb957"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "9817b607b2f74e2ff4fa4ee9215b4545"
server_key_iv: "dd28250a187a02fb2acbfb13b3a9dfe5"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.106325 25206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:38.128777 25206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:38.131016 25206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:38.131124 25206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:38.134964 25206 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:46791
I20260808 07:14:38.135044 25306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:46791 every 8 connection(s)
I20260808 07:14:38.135401 25206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:38.136329 25307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:38.138635 25307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957: Bootstrap starting.
I20260808 07:14:38.139264 25307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:38.139570 25307 log.cc:824] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:38.140291 25307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957: No bootstrap required, opened a new log
I20260808 07:14:38.142088 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 25206
I20260808 07:14:38.141981 25307 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8fa640fa7d1406aabbeebb134aeb957" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46791 } }
I20260808 07:14:38.142155 25307 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:38.142195 25307 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8fa640fa7d1406aabbeebb134aeb957, State: Initialized, Role: FOLLOWER
I20260808 07:14:38.142215 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:38.142351 25307 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8fa640fa7d1406aabbeebb134aeb957" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46791 } }
I20260808 07:14:38.142431 30937 external_mini_cluster.cc:1428] Setting key b23d9c2d98dd6405ded064c30b716f6f
I20260808 07:14:38.142434 25307 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:38.142469 25307 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:38.142534 25307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:38.143553 25307 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8fa640fa7d1406aabbeebb134aeb957" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46791 } }
I20260808 07:14:38.143685 25307 leader_election.cc:302] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8fa640fa7d1406aabbeebb134aeb957; no voters:
I20260808 07:14:38.143882 25307 leader_election.cc:288] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:38.143913 25314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:38.144044 25314 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [term 1 LEADER]: Becoming Leader. State: Replica: c8fa640fa7d1406aabbeebb134aeb957, State: Running, Role: LEADER
I20260808 07:14:38.144147 25307 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:38.144125 25314 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8fa640fa7d1406aabbeebb134aeb957" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46791 } }
I20260808 07:14:38.144531 25314 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8fa640fa7d1406aabbeebb134aeb957. Latest consensus state: current_term: 1 leader_uuid: "c8fa640fa7d1406aabbeebb134aeb957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8fa640fa7d1406aabbeebb134aeb957" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46791 } } }
I20260808 07:14:38.144655 25314 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:38.144896 25317 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8fa640fa7d1406aabbeebb134aeb957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8fa640fa7d1406aabbeebb134aeb957" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46791 } } }
I20260808 07:14:38.145007 25317 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:38.145107 25325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:38.145656 25325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:38.147807 25325 catalog_manager.cc:1347] Generated new cluster ID: f76542a352144bcba6695aa3f738b363
I20260808 07:14:38.147866 25325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:14:38 dist-test-slave-znh6 krb5kdc[25189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:38.154265 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.142934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49340 (local address 127.30.54.126:46791)
0808 07:14:38.143215 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.143223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.143247 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:14:38.143349 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.143354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.143393 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.143531 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:38.144307 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.146553 (+ 2246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.147587 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.148177 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.148292 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.150748 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:38.150751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:38.150752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:38.150761 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:38.152459 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.152833 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.152836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.152837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.152949 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.153159 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.153160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.153160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.153389 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:38.153645 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.153775 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.153998 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":181,"threads_started":1}
I20260808 07:14:38.170501 25325 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:38.171170 25325 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:38.178473 25325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957: Generated new TSK 0
I20260808 07:14:38.178644 25325 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:14:38.207927 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:46791
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:38.288179 25349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:38.288370 25349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:38.288393 25349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:38.289502 25349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:38.289543 25349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:38.289593 25349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:14:38.291013 25349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:46791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:38.291255 25349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:38.291481 25349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:38.296492 25359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:38.296481 25357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:38.296608 25349 server_base.cc:1034] running on GCE node
W20260808 07:14:38.296484 25355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:38.296923 25349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:38.297184 25349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:38.298341 25349 hybrid_clock.cc:648] HybridClock initialized: now 1786173278298322 us; error 39 us; skew 500 ppm
Aug 08 07:14:38 dist-test-slave-znh6 krb5kdc[25189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173278, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:38.300791 25349 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:14:38.301242 25349 webserver.cc:460] Webserver started at http://127.30.54.65:33091/ using document root <none> and password file <none>
I20260808 07:14:38.301469 25349 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:38.301537 25349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:38.301688 25349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:38.302731 25349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "906f611062a64a37b19f81920d55a353"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "a770b91faeb9487ccb3c43dfb25529e4"
server_key_iv: "ad9e98d671bc0385d781e6848f529bbd"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.303167 25349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "906f611062a64a37b19f81920d55a353"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "a770b91faeb9487ccb3c43dfb25529e4"
server_key_iv: "ad9e98d671bc0385d781e6848f529bbd"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.304483 25349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:38.305188 25369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.305352 25349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:38.305452 25349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "906f611062a64a37b19f81920d55a353"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "a770b91faeb9487ccb3c43dfb25529e4"
server_key_iv: "ad9e98d671bc0385d781e6848f529bbd"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.305531 25349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:38.320060 25349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:38.321703 25349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:38.321857 25349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:38.322074 25349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:38.322371 25349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:38.322427 25349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.322459 25349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:38.322482 25349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.328624 25349 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:45839
I20260808 07:14:38.328675 25511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:45839 every 8 connection(s)
I20260808 07:14:38.328961 25349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:38.332687 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 25349
I20260808 07:14:38.332772 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:14:38.332996 30937 external_mini_cluster.cc:1428] Setting key 8d5a933584936256e11669f5987f03ce
Aug 08 07:14:38 dist-test-slave-znh6 krb5kdc[25189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173278, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:38.338909 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.329625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:36497 (local address 127.30.54.126:46791)
0808 07:14:38.329705 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.329708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.329927 (+ 219us) server_negotiation.cc:407] Connection header received
0808 07:14:38.330442 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.330443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.330456 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.330529 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:38.331691 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.332216 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.333389 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.333679 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.333754 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.335917 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:38.335918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:38.335918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:38.335926 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:38.337356 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.337782 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.337783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.337783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.337848 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.338110 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.338111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.338111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.338228 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:38.338427 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.338706 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.338793 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:14:38.339124 25515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.329600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:46791 (local address 127.30.54.65:36497)
0808 07:14:38.329807 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:38.329840 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:38.330327 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:38.330690 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:38.330694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:38.331386 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:38.331568 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.331573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.332332 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.332333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.333315 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.333318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.333764 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.333766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.333891 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.334379 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:38.334387 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:38.335804 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:38.337462 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:38.337470 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:38.337473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:38.337701 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:38.337921 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:38.337922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:38.337922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:38.338038 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:38.338417 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:38.338425 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:38.338567 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 07:14:38.338682 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 07:14:38.338883 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":120,"threads_started":1}
I20260808 07:14:38.339694 25513 heartbeater.cc:347] Connected to a master server at 127.30.54.126:46791
I20260808 07:14:38.339809 25513 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:38.339982 25513 heartbeater.cc:511] Master 127.30.54.126:46791 requested a full tablet report, sending...
I20260808 07:14:38.340544 25258 ts_manager.cc:194] Registered new tserver with Master: 906f611062a64a37b19f81920d55a353 (127.30.54.65:45839)
I20260808 07:14:38.341521 25258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:36497
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:14:38.359174 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:46791
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:38.447008 25520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:38.447201 25520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:38.447223 25520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:38.448352 25520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:38.448395 25520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:38.448444 25520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:14:38.449864 25520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:46791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:38.450099 25520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:38.450341 25520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:38.455675 25529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:38.455669 25526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:38.455809 25531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:38.455964 25520 server_base.cc:1034] running on GCE node
I20260808 07:14:38.456174 25520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:38.456409 25520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:38.457576 25520 hybrid_clock.cc:648] HybridClock initialized: now 1786173278457564 us; error 29 us; skew 500 ppm
Aug 08 07:14:38 dist-test-slave-znh6 krb5kdc[25189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173278, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:38.460160 25520 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:14:38.460582 25520 webserver.cc:460] Webserver started at http://127.30.54.66:38643/ using document root <none> and password file <none>
I20260808 07:14:38.460805 25520 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:38.460875 25520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:38.461036 25520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:38.462031 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "9bca65a748fd4120adeabd9de1777920"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "fc84e488f528c78db1b3a9feed415ba8"
server_key_iv: "999bb6e9e66b2d163ca5202f0ccf747c"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.462396 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "9bca65a748fd4120adeabd9de1777920"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "fc84e488f528c78db1b3a9feed415ba8"
server_key_iv: "999bb6e9e66b2d163ca5202f0ccf747c"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.463851 25520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:14:38.464581 25541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.464758 25520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:38.464857 25520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "9bca65a748fd4120adeabd9de1777920"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "fc84e488f528c78db1b3a9feed415ba8"
server_key_iv: "999bb6e9e66b2d163ca5202f0ccf747c"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.464936 25520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:38.477648 25520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:38.479894 25520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:38.480078 25520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:38.480311 25520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:38.480610 25520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:38.480664 25520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.480700 25520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:38.480726 25520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.486446 25520 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:46793
I20260808 07:14:38.486524 25702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:46793 every 8 connection(s)
I20260808 07:14:38.486853 25520 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:14:38 dist-test-slave-znh6 krb5kdc[25189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173278, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:38.494202 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 25520
I20260808 07:14:38.494323 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:14:38.494532 30937 external_mini_cluster.cc:1428] Setting key d6aecea2df02eda79b9983d4c76b7182
I20260808 07:14:38.495900 25706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.487469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:46791 (local address 127.30.54.66:54339)
0808 07:14:38.487645 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:38.487678 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:38.488148 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:38.488404 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:38.488408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:38.488947 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:38.489116 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.489121 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.489835 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.489836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.490613 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.490616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.491098 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.491099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.491218 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.491601 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:38.491608 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:38.492888 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:38.494404 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:38.494412 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:38.494414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:38.494720 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:38.495012 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:38.495013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:38.495013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:38.495151 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:38.495436 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:38.495442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:38.495553 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 07:14:38.495640 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:14:38.495754 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":116,"threads_started":1}
I20260808 07:14:38.495908 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.487521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:54339 (local address 127.30.54.126:46791)
0808 07:14:38.487610 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.487612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.487768 (+ 156us) server_negotiation.cc:407] Connection header received
0808 07:14:38.488255 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.488256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.488269 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.488332 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:38.489219 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.489737 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.490720 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.491005 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.491077 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.493000 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:38.493002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:38.493002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:38.493009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:38.494311 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.494864 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.494866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.494866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.494930 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.495224 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.495225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.495225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.495363 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:38.495460 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.495685 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.495777 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:14:38.496490 25703 heartbeater.cc:347] Connected to a master server at 127.30.54.126:46791
I20260808 07:14:38.496588 25703 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:38.496767 25703 heartbeater.cc:511] Master 127.30.54.126:46791 requested a full tablet report, sending...
I20260808 07:14:38.497152 25258 ts_manager.cc:194] Registered new tserver with Master: 9bca65a748fd4120adeabd9de1777920 (127.30.54.66:46793)
I20260808 07:14:38.497558 25258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:54339
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:14:38.521883 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:46791
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:38.605890 25711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:38.606125 25711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:38.606168 25711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:38.607403 25711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:38.607470 25711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:38.607532 25711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:14:38.609071 25711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:46791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:38.609328 25711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:38.609573 25711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:38.614950 25719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:38.614940 25718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:38.614936 25721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:38.614970 25711 server_base.cc:1034] running on GCE node
I20260808 07:14:38.615379 25711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:38.615610 25711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:38.616763 25711 hybrid_clock.cc:648] HybridClock initialized: now 1786173278616746 us; error 34 us; skew 500 ppm
Aug 08 07:14:38 dist-test-slave-znh6 krb5kdc[25189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173278, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:38.619374 25711 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:14:38.619817 25711 webserver.cc:460] Webserver started at http://127.30.54.67:43843/ using document root <none> and password file <none>
I20260808 07:14:38.620000 25711 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:38.620041 25711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:38.620131 25711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:38.621083 25711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "9ee309792964482a9cfde6d646590129"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "20186a6efa9aa59160fdb7c87166f84c"
server_key_iv: "c180e82690174893559837c2c66c6f44"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.621381 25711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "9ee309792964482a9cfde6d646590129"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "20186a6efa9aa59160fdb7c87166f84c"
server_key_iv: "c180e82690174893559837c2c66c6f44"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.622648 25711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:14:38.623421 25730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.623603 25711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.623708 25711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "9ee309792964482a9cfde6d646590129"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "20186a6efa9aa59160fdb7c87166f84c"
server_key_iv: "c180e82690174893559837c2c66c6f44"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.623782 25711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:38.633862 25711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:38.636198 25711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:38.636361 25711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:38.636590 25711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:38.636915 25711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:38.636971 25711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.637020 25711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:38.637048 25711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.643113 25711 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:38705
I20260808 07:14:38.643169 25891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:38705 every 8 connection(s)
I20260808 07:14:38.643522 25711 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:14:38.646389 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 25711
I20260808 07:14:38.646521 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:14:38.646725 30937 external_mini_cluster.cc:1428] Setting key 0a324044d0b08fbb4ad79de25b4cd266
Aug 08 07:14:38 dist-test-slave-znh6 krb5kdc[25189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173278, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:38.653198 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.644289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:59785 (local address 127.30.54.126:46791)
0808 07:14:38.644332 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.644334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.644548 (+ 214us) server_negotiation.cc:407] Connection header received
0808 07:14:38.645173 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.645174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.645186 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.645250 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:38.646263 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.646966 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.647991 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.648208 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.648286 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.650429 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:38.650430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:38.650431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:38.650437 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:38.652022 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.652370 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.652371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.652371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.652422 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.652636 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.652636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.652637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.652737 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:38.652788 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.653018 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.653078 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:14:38.653476 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.644245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:46791 (local address 127.30.54.67:59785)
0808 07:14:38.644415 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:38.644457 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:38.645061 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:38.645320 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:38.645324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:38.645893 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:38.646161 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.646166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.647066 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.647068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.647878 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.647880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.648293 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.648295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.648430 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.648952 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:38.648960 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:38.650344 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:38.652091 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:38.652096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:38.652098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:38.652306 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:38.652483 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:38.652483 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:38.652484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:38.652579 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:38.652792 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:38.652796 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:38.652901 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 07:14:38.652988 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:14:38.653121 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260808 07:14:38.654342 25893 heartbeater.cc:347] Connected to a master server at 127.30.54.126:46791
I20260808 07:14:38.654448 25893 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:38.654639 25893 heartbeater.cc:511] Master 127.30.54.126:46791 requested a full tablet report, sending...
I20260808 07:14:38.655045 25258 ts_manager.cc:194] Registered new tserver with Master: 9ee309792964482a9cfde6d646590129 (127.30.54.67:38705)
I20260808 07:14:38.655623 25258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:59785
I20260808 07:14:38.658296 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:14:38.668999 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.662534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49342 (local address 127.30.54.126:46791)
0808 07:14:38.662596 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.662598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.662993 (+ 395us) server_negotiation.cc:407] Connection header received
0808 07:14:38.663036 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.663037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.663047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.663100 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:38.663640 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.664118 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.664802 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.664975 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.665026 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.666265 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:38.666266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:38.666267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:38.666276 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:38.667823 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.668182 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.668182 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.668183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.668234 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:38.668417 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:38.668418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:38.668418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:38.668551 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:38.668629 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.668814 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.668891 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:14:38.670399 25258 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:14:38.672118 25258 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:14:38.678321 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.674306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46793 (local address 127.0.0.1:34310)
0808 07:14:38.674762 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:38.674771 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:38.674848 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:38.675160 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:38.675163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:38.675169 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:38.675266 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.675268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.676151 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.676152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.677100 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.677102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.678012 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.678013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.678107 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.678125 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 07:14:38.678194 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:14:38.678248 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260808 07:14:38.678344 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.674558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.30.54.66:46793)
0808 07:14:38.674760 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.674766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.674810 (+ 44us) server_negotiation.cc:407] Connection header received
0808 07:14:38.674958 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.674961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.674979 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.675053 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:38.675358 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.676055 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.677194 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.677935 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.678009 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.678071 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.678213 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.678261 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":95,"threads_started":1}
I20260808 07:14:38.679103 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.674276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:38705 (local address 127.0.0.1:40190)
0808 07:14:38.674603 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:38.674615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:38.674672 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:38.675162 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:38.675163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:38.675165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:38.675301 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.675304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.676690 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.676691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.677885 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.677888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.678870 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.678871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.678939 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.678943 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:38.678988 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:14:38.679027 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":81,"thread_start_us":81,"threads_started":1}
I20260808 07:14:38.679283 25906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.674276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:45839 (local address 127.0.0.1:37498)
0808 07:14:38.674547 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:38.674589 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:38.674643 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:38.675553 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:38.675553 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:38.675555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:38.675592 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.675594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.676948 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.676949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.677724 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.677726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.679019 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.679020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.679085 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.679089 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:38.679152 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 07:14:38.679194 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":81,"threads_started":1}
I20260808 07:14:38.679294 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.674403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.30.54.65:45839)
0808 07:14:38.675311 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.675316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.675326 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:14:38.675370 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.675372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.675388 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.675465 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:38.675674 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.676834 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.678104 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.678932 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.679006 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.679067 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.679164 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.679215 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":102,"threads_started":1}
I20260808 07:14:38.679276 25610 tablet_service.cc:1467] Processing CreateTablet for tablet 96dc81d49d3c495ea07aed9f2de222f4 (DEFAULT_TABLE table=test-table [id=836daac89cdd44519e996b4ae78cfb1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:38.679433 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.674751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.30.54.67:38705)
0808 07:14:38.674983 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.674987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.675005 (+ 18us) server_negotiation.cc:407] Connection header received
0808 07:14:38.675065 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.675067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.675082 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.675165 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:38.675393 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.676556 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.677992 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.678696 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.678769 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.679261 (+ 492us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.679302 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.679346 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":130,"threads_started":1}
I20260808 07:14:38.679692 25610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dc81d49d3c495ea07aed9f2de222f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:38.680065 25804 tablet_service.cc:1467] Processing CreateTablet for tablet 96dc81d49d3c495ea07aed9f2de222f4 (DEFAULT_TABLE table=test-table [id=836daac89cdd44519e996b4ae78cfb1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:38.680013 25436 tablet_service.cc:1467] Processing CreateTablet for tablet 96dc81d49d3c495ea07aed9f2de222f4 (DEFAULT_TABLE table=test-table [id=836daac89cdd44519e996b4ae78cfb1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:38.680315 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dc81d49d3c495ea07aed9f2de222f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:38.680383 25436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96dc81d49d3c495ea07aed9f2de222f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:38.682706 25913 tablet_bootstrap.cc:492] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: Bootstrap starting.
I20260808 07:14:38.683430 25914 tablet_bootstrap.cc:492] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: Bootstrap starting.
I20260808 07:14:38.683459 25913 tablet_bootstrap.cc:654] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:38.683741 25916 tablet_bootstrap.cc:492] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353: Bootstrap starting.
I20260808 07:14:38.684098 25914 tablet_bootstrap.cc:654] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:38.684092 25913 log.cc:824] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:38.684515 25916 tablet_bootstrap.cc:654] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:38.684696 25914 log.cc:824] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:38.685012 25913 tablet_bootstrap.cc:492] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: No bootstrap required, opened a new log
I20260808 07:14:38.685060 25916 log.cc:824] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:38.685091 25913 ts_tablet_manager.cc:1397] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 07:14:38.685490 25914 tablet_bootstrap.cc:492] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: No bootstrap required, opened a new log
I20260808 07:14:38.685566 25914 ts_tablet_manager.cc:1397] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:14:38.685842 25916 tablet_bootstrap.cc:492] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353: No bootstrap required, opened a new log
I20260808 07:14:38.685918 25916 ts_tablet_manager.cc:1397] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:14:38.687062 25913 raft_consensus.cc:348] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } }
I20260808 07:14:38.687245 25913 raft_consensus.cc:374] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:38.687284 25913 raft_consensus.cc:729] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ee309792964482a9cfde6d646590129, State: Initialized, Role: FOLLOWER
I20260808 07:14:38.687311 25914 raft_consensus.cc:348] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } }
I20260808 07:14:38.687426 25913 consensus_queue.cc:260] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } }
I20260808 07:14:38.687505 25914 raft_consensus.cc:374] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:38.687551 25914 raft_consensus.cc:729] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bca65a748fd4120adeabd9de1777920, State: Initialized, Role: FOLLOWER
I20260808 07:14:38.687644 25914 consensus_queue.cc:260] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } }
I20260808 07:14:38.687695 25913 ts_tablet_manager.cc:1428] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:14:38.687844 25893 heartbeater.cc:503] Master 127.30.54.126:46791 was elected leader, sending a full tablet report...
I20260808 07:14:38.687903 25914 ts_tablet_manager.cc:1428] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 07:14:38.688021 25703 heartbeater.cc:503] Master 127.30.54.126:46791 was elected leader, sending a full tablet report...
I20260808 07:14:38.687927 25916 raft_consensus.cc:348] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } }
I20260808 07:14:38.688083 25916 raft_consensus.cc:374] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:38.688117 25916 raft_consensus.cc:729] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906f611062a64a37b19f81920d55a353, State: Initialized, Role: FOLLOWER
I20260808 07:14:38.688241 25916 consensus_queue.cc:260] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } }
I20260808 07:14:38.688546 25916 ts_tablet_manager.cc:1428] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:38.688625 25513 heartbeater.cc:503] Master 127.30.54.126:46791 was elected leader, sending a full tablet report...
W20260808 07:14:38.737421 25705 tablet.cc:2369] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:14:38.749634 25924 raft_consensus.cc:482] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:38.749737 25924 raft_consensus.cc:504] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } }
I20260808 07:14:38.750170 25924 leader_election.cc:288] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ee309792964482a9cfde6d646590129 (127.30.54.67:38705), 9bca65a748fd4120adeabd9de1777920 (127.30.54.66:46793)
I20260808 07:14:38.754184 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.750457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:40405 (local address 127.30.54.67:38705)
0808 07:14:38.750512 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.750514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.750673 (+ 159us) server_negotiation.cc:407] Connection header received
0808 07:14:38.750701 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.750702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.750709 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.750777 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:38.750963 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.751626 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.753105 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.753692 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.753766 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.753774 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.754071 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.754112 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:14:38.754186 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.750426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:46521 (local address 127.30.54.66:46793)
0808 07:14:38.750482 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.750485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.750495 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:14:38.750527 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.750529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.750544 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.750599 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:38.750796 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.751577 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.753105 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.753688 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.753766 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.753774 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.754071 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.754112 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:14:38.754216 25515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.750343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46793 (local address 127.30.54.65:46521)
0808 07:14:38.750404 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:38.750416 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:38.750465 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:38.750610 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:38.750611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:38.750613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:38.750695 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.750697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.751682 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.751683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.753008 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.753011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.753884 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.753885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.753981 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.753983 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:14:38.754052 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:14:38.754116 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":79}
I20260808 07:14:38.754225 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.750359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:38705 (local address 127.30.54.65:40405)
0808 07:14:38.750600 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:38.750609 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:38.750659 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:38.750802 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:38.750802 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:38.750804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:38.750894 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.750895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.751700 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.751700 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.753008 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:38.753011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.753884 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.753885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:38.753976 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.753981 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:14:38.754052 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:14:38.754116 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":133,"thread_start_us":132,"threads_started":1}
I20260808 07:14:38.754562 25635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dc81d49d3c495ea07aed9f2de222f4" candidate_uuid: "906f611062a64a37b19f81920d55a353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bca65a748fd4120adeabd9de1777920" is_pre_election: true
I20260808 07:14:38.754683 25635 raft_consensus.cc:2454] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906f611062a64a37b19f81920d55a353 in term 0.
I20260808 07:14:38.754909 25370 leader_election.cc:302] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 906f611062a64a37b19f81920d55a353, 9bca65a748fd4120adeabd9de1777920; no voters:
I20260808 07:14:38.754922 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dc81d49d3c495ea07aed9f2de222f4" candidate_uuid: "906f611062a64a37b19f81920d55a353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ee309792964482a9cfde6d646590129" is_pre_election: true
I20260808 07:14:38.755029 25924 raft_consensus.cc:2781] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:14:38.755088 25924 raft_consensus.cc:482] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:14:38.755084 25829 raft_consensus.cc:2454] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906f611062a64a37b19f81920d55a353 in term 0.
I20260808 07:14:38.755119 25924 raft_consensus.cc:3037] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:38.755811 25924 raft_consensus.cc:504] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } }
I20260808 07:14:38.755955 25924 leader_election.cc:288] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [CANDIDATE]: Term 1 election: Requested vote from peers 9ee309792964482a9cfde6d646590129 (127.30.54.67:38705), 9bca65a748fd4120adeabd9de1777920 (127.30.54.66:46793)
I20260808 07:14:38.756074 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dc81d49d3c495ea07aed9f2de222f4" candidate_uuid: "906f611062a64a37b19f81920d55a353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ee309792964482a9cfde6d646590129"
I20260808 07:14:38.756157 25829 raft_consensus.cc:3037] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:38.756172 25635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96dc81d49d3c495ea07aed9f2de222f4" candidate_uuid: "906f611062a64a37b19f81920d55a353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bca65a748fd4120adeabd9de1777920"
I20260808 07:14:38.756242 25635 raft_consensus.cc:3037] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:38.756922 25635 raft_consensus.cc:2454] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906f611062a64a37b19f81920d55a353 in term 1.
I20260808 07:14:38.756933 25829 raft_consensus.cc:2454] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906f611062a64a37b19f81920d55a353 in term 1.
I20260808 07:14:38.757078 25370 leader_election.cc:302] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 906f611062a64a37b19f81920d55a353, 9bca65a748fd4120adeabd9de1777920; no voters:
I20260808 07:14:38.757162 25924 raft_consensus.cc:2781] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:38.757308 25924 raft_consensus.cc:686] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 1 LEADER]: Becoming Leader. State: Replica: 906f611062a64a37b19f81920d55a353, State: Running, Role: LEADER
I20260808 07:14:38.757411 25924 consensus_queue.cc:237] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } }
I20260808 07:14:38.758179 25257 catalog_manager.cc:5495] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 reported cstate change: term changed from 0 to 1, leader changed from <none> to 906f611062a64a37b19f81920d55a353 (127.30.54.65). New cstate: current_term: 1 leader_uuid: "906f611062a64a37b19f81920d55a353" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906f611062a64a37b19f81920d55a353" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 45839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 } health_report { overall_health: UNKNOWN } } }
I20260808 07:14:38.774016 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.771626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.30.54.65:45839)
0808 07:14:38.771673 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.771675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.771684 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:38.771769 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.771770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.771792 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.771868 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:14:38.772029 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.772471 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.773232 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.773423 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.773505 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.773675 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:14:38.773763 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:14:38.773866 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.773926 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.773938 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:14:38.775717 25829 raft_consensus.cc:1261] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [term 1 FOLLOWER]: Refusing update from remote peer 906f611062a64a37b19f81920d55a353: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:38.775719 25635 raft_consensus.cc:1261] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [term 1 FOLLOWER]: Refusing update from remote peer 906f611062a64a37b19f81920d55a353: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:38.776064 25924 consensus_queue.cc:1035] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ee309792964482a9cfde6d646590129" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 38705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:38.776196 25926 consensus_queue.cc:1035] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bca65a748fd4120adeabd9de1777920" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:38.778738 25930 mvcc.cc:205] Tried to move back new op lower bound from 7316165749862670336 to 7316165749790928896. Current Snapshot: MvccSnapshot[applied={T|T < 7316165749862670336}]
I20260808 07:14:38.780473 25931 mvcc.cc:205] Tried to move back new op lower bound from 7316165749862670336 to 7316165749790928896. Current Snapshot: MvccSnapshot[applied={T|T < 7316165749862670336 or (T in {7316165749862670336})}]
I20260808 07:14:38.780800 25258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:38.780890 25258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:38.781744 25258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8fa640fa7d1406aabbeebb134aeb957: Sending DeleteTablet for 3 replicas of tablet 96dc81d49d3c495ea07aed9f2de222f4
I20260808 07:14:38.782109 25436 tablet_service.cc:1514] Processing DeleteTablet for tablet 96dc81d49d3c495ea07aed9f2de222f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:38 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:37498
I20260808 07:14:38.782116 25610 tablet_service.cc:1514] Processing DeleteTablet for tablet 96dc81d49d3c495ea07aed9f2de222f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:38 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:34310
I20260808 07:14:38.782130 25804 tablet_service.cc:1514] Processing DeleteTablet for tablet 96dc81d49d3c495ea07aed9f2de222f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:38 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:40190
I20260808 07:14:38.782354 25946 tablet_replica.cc:330] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: stopping tablet replica
I20260808 07:14:38.782366 25943 tablet_replica.cc:330] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353: stopping tablet replica
I20260808 07:14:38.782459 25946 raft_consensus.cc:2229] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:38.782469 25943 raft_consensus.cc:2229] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:14:38.782553 25946 raft_consensus.cc:2258] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:38.782565 25943 raft_consensus.cc:2258] T 96dc81d49d3c495ea07aed9f2de222f4 P 906f611062a64a37b19f81920d55a353 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:38.782769 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 25349
I20260808 07:14:38.782785 25945 tablet_replica.cc:330] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: stopping tablet replica
I20260808 07:14:38.782888 25945 raft_consensus.cc:2229] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:38.782965 25945 raft_consensus.cc:2258] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:38.783524 25946 ts_tablet_manager.cc:1905] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:38.786657 25945 ts_tablet_manager.cc:1905] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:38.788641 25945 ts_tablet_manager.cc:1918] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:38.788703 25945 log.cc:1196] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/96dc81d49d3c495ea07aed9f2de222f4
I20260808 07:14:38.788949 25945 ts_tablet_manager.cc:1939] T 96dc81d49d3c495ea07aed9f2de222f4 P 9bca65a748fd4120adeabd9de1777920: Deleting consensus metadata
I20260808 07:14:38.789541 25230 catalog_manager.cc:4841] TS 9bca65a748fd4120adeabd9de1777920 (127.30.54.66:46793): tablet 96dc81d49d3c495ea07aed9f2de222f4 (table test-table [id=836daac89cdd44519e996b4ae78cfb1b]) successfully deleted
I20260808 07:14:38.790587 25946 ts_tablet_manager.cc:1918] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:38.790658 25946 log.cc:1196] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/96dc81d49d3c495ea07aed9f2de222f4
I20260808 07:14:38.790961 25946 ts_tablet_manager.cc:1939] T 96dc81d49d3c495ea07aed9f2de222f4 P 9ee309792964482a9cfde6d646590129: Deleting consensus metadata
I20260808 07:14:38.791399 25233 catalog_manager.cc:4841] TS 9ee309792964482a9cfde6d646590129 (127.30.54.67:38705): tablet 96dc81d49d3c495ea07aed9f2de222f4 (table test-table [id=836daac89cdd44519e996b4ae78cfb1b]) successfully deleted
W20260808 07:14:38.796097 25231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:45839 (error 108)
I20260808 07:14:38.796597 25906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.796455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:45839 (local address 127.0.0.1:37510)
0808 07:14:38.796518 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:38.796560 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:45839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:14:38.796666 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 25520
W20260808 07:14:38.796741 25231 catalog_manager.cc:4584] TS 906f611062a64a37b19f81920d55a353 (127.30.54.65:45839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96dc81d49d3c495ea07aed9f2de222f4: Network error: Client connection negotiation failed: client connection to 127.30.54.65:45839: connect: Connection refused (error 111)
I20260808 07:14:38.803094 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 25711
I20260808 07:14:38.817260 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 25206
2026-08-08T07:14:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2161 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T07:14:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:38Z Disabled control of system clock
I20260808 07:14:38.850248 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34467
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:43833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:34467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:14:38.931782 25954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:38.931977 25954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:38.931998 25954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:38.933043 25954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:38.933079 25954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:38.933091 25954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:38.933104 25954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:38.933115 25954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:38.934435 25954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:34467
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:38.934621 25954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:38.934900 25954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:38.939975 25960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:38.940145 25954 server_base.cc:1034] running on GCE node
W20260808 07:14:38.940026 25965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:38.940040 25962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:38.940642 25954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:38.940984 25954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:38.942173 25954 hybrid_clock.cc:648] HybridClock initialized: now 1786173278942150 us; error 36 us; skew 500 ppm
I20260808 07:14:38.948299 25954 webserver.cc:460] Webserver started at http://127.30.54.126:39745/ using document root <none> and password file <none>
I20260808 07:14:38.948642 25954 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:38.948722 25954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:38.948876 25954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:38.950096 25954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "5b59133238cf451bb4cc76e5f6a18f36"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "b87b3f51e546b761b624ee46b3e6a76c"
server_key_iv: "755ea09c109717b9e1b50cbd47ce524d"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.950387 25954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "5b59133238cf451bb4cc76e5f6a18f36"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "b87b3f51e546b761b624ee46b3e6a76c"
server_key_iv: "755ea09c109717b9e1b50cbd47ce524d"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.951694 25954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:14:38.952381 25975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.952574 25954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:38.952644 25954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "5b59133238cf451bb4cc76e5f6a18f36"
format_stamp: "Formatted at 2026-08-08 07:14:38 on dist-test-slave-znh6"
server_key: "b87b3f51e546b761b624ee46b3e6a76c"
server_key_iv: "755ea09c109717b9e1b50cbd47ce524d"
server_key_version: "encryptionkey@0"
I20260808 07:14:38.952695 25954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:38.969700 25954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:38.969933 25954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:38.970031 25954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:38.973948 25954 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:34467
I20260808 07:14:38.974033 26050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:34467 every 8 connection(s)
I20260808 07:14:38.974365 25954 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:38.974746 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 25954
I20260808 07:14:38.974884 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:38.975116 30937 external_mini_cluster.cc:1428] Setting key 9251157bcf6c9d4b9c0ec46c99cc8d46
I20260808 07:14:38.975661 26051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:38.979005 26051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36: Bootstrap starting.
I20260808 07:14:38.979704 26051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:38.980048 26051 log.cc:824] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:38.980727 26051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36: No bootstrap required, opened a new log
I20260808 07:14:38.981156 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:38.975655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33486 (local address 127.30.54.126:34467)
0808 07:14:38.975900 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:38.975909 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:38.975938 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:14:38.976473 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:38.976482 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:38.976562 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:38.976713 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:38.976918 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.978255 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.979077 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:38.979694 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:38.979844 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:38.980531 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:38.980535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:38.980535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:38.980702 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:38.980734 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:38.980744 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:38.980822 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:38.980901 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:14:38.980983 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":104,"threads_started":1}
I20260808 07:14:38.982767 26051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b59133238cf451bb4cc76e5f6a18f36" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34467 } }
I20260808 07:14:38.982972 26051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:38.983047 26051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b59133238cf451bb4cc76e5f6a18f36, State: Initialized, Role: FOLLOWER
I20260808 07:14:38.983170 26051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b59133238cf451bb4cc76e5f6a18f36" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34467 } }
I20260808 07:14:38.983268 26051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:38.983325 26051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:38.983374 26051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:38.983919 26051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b59133238cf451bb4cc76e5f6a18f36" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34467 } }
I20260808 07:14:38.984031 26051 leader_election.cc:302] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b59133238cf451bb4cc76e5f6a18f36; no voters:
I20260808 07:14:38.984205 26051 leader_election.cc:288] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:38.984266 26056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:38.984458 26056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [term 1 LEADER]: Becoming Leader. State: Replica: 5b59133238cf451bb4cc76e5f6a18f36, State: Running, Role: LEADER
I20260808 07:14:38.984481 26051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:38.984547 26056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b59133238cf451bb4cc76e5f6a18f36" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34467 } }
I20260808 07:14:38.984944 26058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b59133238cf451bb4cc76e5f6a18f36. Latest consensus state: current_term: 1 leader_uuid: "5b59133238cf451bb4cc76e5f6a18f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b59133238cf451bb4cc76e5f6a18f36" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34467 } } }
I20260808 07:14:38.985045 26058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:38.985213 26057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b59133238cf451bb4cc76e5f6a18f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b59133238cf451bb4cc76e5f6a18f36" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34467 } } }
I20260808 07:14:38.985260 26070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:38.985463 26057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:38.985925 26070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:38.987407 26070 catalog_manager.cc:1347] Generated new cluster ID: c5e82837bf6944fd9c00f328905ef97c
I20260808 07:14:38.987469 26070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:38.993069 26070 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:38.993583 26070 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:38.995600 26070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36: Generated new TSK 0
I20260808 07:14:38.995766 26070 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:38.999698 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34467
--builtin_ntp_servers=127.30.54.84:43833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:14:39.082278 26083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:39.082479 26083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:39.082500 26083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:39.084235 26083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:39.084285 26083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:39.084344 26083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:14:39.085916 26083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:39.086150 26083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:39.086459 26083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:39.091482 26090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:39.091463 26092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:39.091663 26083 server_base.cc:1034] running on GCE node
W20260808 07:14:39.091801 26089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:39.092031 26083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:39.092268 26083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:39.093426 26083 hybrid_clock.cc:648] HybridClock initialized: now 1786173279093404 us; error 41 us; skew 500 ppm
I20260808 07:14:39.094691 26083 webserver.cc:460] Webserver started at http://127.30.54.65:38063/ using document root <none> and password file <none>
I20260808 07:14:39.094981 26083 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:39.095055 26083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:39.095192 26083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:39.096208 26083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "f4220d86ccb44df992ad15d9a91a4aa7"
format_stamp: "Formatted at 2026-08-08 07:14:39 on dist-test-slave-znh6"
server_key: "ec58c1ca02f55d5190b56de38ec8c43b"
server_key_iv: "d570d7a0c0bd03826d8a52f9abca4735"
server_key_version: "encryptionkey@0"
I20260808 07:14:39.096582 26083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "f4220d86ccb44df992ad15d9a91a4aa7"
format_stamp: "Formatted at 2026-08-08 07:14:39 on dist-test-slave-znh6"
server_key: "ec58c1ca02f55d5190b56de38ec8c43b"
server_key_iv: "d570d7a0c0bd03826d8a52f9abca4735"
server_key_version: "encryptionkey@0"
I20260808 07:14:39.097945 26083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:14:39.098719 26099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.098939 26083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:39.099040 26083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "f4220d86ccb44df992ad15d9a91a4aa7"
format_stamp: "Formatted at 2026-08-08 07:14:39 on dist-test-slave-znh6"
server_key: "ec58c1ca02f55d5190b56de38ec8c43b"
server_key_iv: "d570d7a0c0bd03826d8a52f9abca4735"
server_key_version: "encryptionkey@0"
I20260808 07:14:39.099129 26083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:39.115846 26083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:39.116148 26083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:39.116302 26083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:39.116533 26083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:39.116868 26083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:39.116930 26083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.116969 26083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:39.116999 26083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.122881 26083 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:41229
I20260808 07:14:39.122954 26267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:41229 every 8 connection(s)
I20260808 07:14:39.123302 26083 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:39.125367 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 26083
I20260808 07:14:39.125461 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:14:39.125672 30937 external_mini_cluster.cc:1428] Setting key c672ebe028df777bba9f47c9a4e2ee11
I20260808 07:14:39.126628 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34467
--builtin_ntp_servers=127.30.54.84:43833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:14:39.128818 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.124058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:57367 (local address 127.30.54.126:34467)
0808 07:14:39.124108 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.124110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.124380 (+ 270us) server_negotiation.cc:407] Connection header received
0808 07:14:39.124826 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.124827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.124838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.124882 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:39.125599 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.126246 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.127350 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.127566 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.127647 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.128552 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:39.128554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:39.128554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:39.128558 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:39.128573 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:39.128576 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:39.128624 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.128729 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.128760 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:14:39.129079 26271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.124061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34467 (local address 127.30.54.65:57367)
0808 07:14:39.124257 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:39.124290 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:39.124744 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:39.124953 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:39.124956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:39.124969 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:39.125491 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.125499 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.126345 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.126346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.127255 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.127258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.127655 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.127656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.127788 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.128379 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:14:39.128380 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:39.128420 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:14:39.128456 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:39.128643 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:39.128649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:39.128652 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:39.128726 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:14:39.128838 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260808 07:14:39.129660 26269 heartbeater.cc:347] Connected to a master server at 127.30.54.126:34467
I20260808 07:14:39.129741 26269 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:39.129915 26269 heartbeater.cc:511] Master 127.30.54.126:34467 requested a full tablet report, sending...
I20260808 07:14:39.130405 25999 ts_manager.cc:194] Registered new tserver with Master: f4220d86ccb44df992ad15d9a91a4aa7 (127.30.54.65:41229)
I20260808 07:14:39.131062 25999 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.65:57367
W20260808 07:14:39.208192 26272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:39.208379 26272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:39.208401 26272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:39.209524 26272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:39.209564 26272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:39.209589 26272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:14:39.211066 26272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:39.211279 26272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:39.211578 26272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:39.216509 26281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:39.216555 26284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:39.216533 26282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:39.216671 26272 server_base.cc:1034] running on GCE node
I20260808 07:14:39.216871 26272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:39.217054 26272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:39.218189 26272 hybrid_clock.cc:648] HybridClock initialized: now 1786173279218178 us; error 35 us; skew 500 ppm
I20260808 07:14:39.219482 26272 webserver.cc:460] Webserver started at http://127.30.54.66:45713/ using document root <none> and password file <none>
I20260808 07:14:39.219715 26272 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:39.219764 26272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:39.219954 26272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:39.221071 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "6ab0750585064f16a793430500c504fb"
format_stamp: "Formatted at 2026-08-08 07:14:39 on dist-test-slave-znh6"
server_key: "687c62eef13535964524cf66d3fc76fe"
server_key_iv: "0fde20c32bf67ab156da4699a5b8e203"
server_key_version: "encryptionkey@0"
I20260808 07:14:39.221427 26272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "6ab0750585064f16a793430500c504fb"
format_stamp: "Formatted at 2026-08-08 07:14:39 on dist-test-slave-znh6"
server_key: "687c62eef13535964524cf66d3fc76fe"
server_key_iv: "0fde20c32bf67ab156da4699a5b8e203"
server_key_version: "encryptionkey@0"
I20260808 07:14:39.222769 26272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:14:39.223577 26294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.223747 26272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:39.223848 26272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "6ab0750585064f16a793430500c504fb"
format_stamp: "Formatted at 2026-08-08 07:14:39 on dist-test-slave-znh6"
server_key: "687c62eef13535964524cf66d3fc76fe"
server_key_iv: "0fde20c32bf67ab156da4699a5b8e203"
server_key_version: "encryptionkey@0"
I20260808 07:14:39.223927 26272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:39.238922 26272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:39.239190 26272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:39.239342 26272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:39.239563 26272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:39.239934 26272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:39.240000 26272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.240042 26272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:39.240069 26272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.246055 26272 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:40891
I20260808 07:14:39.246090 26452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:40891 every 8 connection(s)
I20260808 07:14:39.246420 26272 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:14:39.251269 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.247159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:35893 (local address 127.30.54.126:34467)
0808 07:14:39.247208 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.247210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.247478 (+ 268us) server_negotiation.cc:407] Connection header received
0808 07:14:39.247912 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.247913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.247931 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.247983 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:39.248591 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.249082 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.250027 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.250201 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.250251 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.251077 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:39.251078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:39.251078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:39.251082 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:39.251093 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:39.251096 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:39.251137 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.251193 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.251212 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:14:39.251502 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.247150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34467 (local address 127.30.54.66:35893)
0808 07:14:39.247362 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:39.247393 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:39.247834 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:39.248031 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:39.248034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:39.248042 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:39.248523 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.248527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.249166 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.249167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.249953 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.249955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.250293 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.250294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.250396 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.250955 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:14:39.250956 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:39.250983 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:14:39.251004 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:39.251137 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:39.251139 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:39.251141 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:14:39.251184 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:14:39.251253 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":95,"threads_started":1}
I20260808 07:14:39.252036 26453 heartbeater.cc:347] Connected to a master server at 127.30.54.126:34467
I20260808 07:14:39.252127 26453 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:39.252259 26453 heartbeater.cc:511] Master 127.30.54.126:34467 requested a full tablet report, sending...
I20260808 07:14:39.252637 25999 ts_manager.cc:194] Registered new tserver with Master: 6ab0750585064f16a793430500c504fb (127.30.54.66:40891)
I20260808 07:14:39.252803 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 26272
I20260808 07:14:39.252929 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:14:39.253103 25999 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.66:35893
I20260808 07:14:39.253124 30937 external_mini_cluster.cc:1428] Setting key 425648c4db1f1fbc6f0ee54cf9d65cd4
I20260808 07:14:39.254069 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34467
--builtin_ntp_servers=127.30.54.84:43833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:14:39.335902 26456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:39.336092 26456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:39.336114 26456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:39.337234 26456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:39.337275 26456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:39.337299 26456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:14:39.338608 26456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:43833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:39.338861 26456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:39.339167 26456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:39.344267 26465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:39.344352 26456 server_base.cc:1034] running on GCE node
W20260808 07:14:39.344333 26466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:39.344264 26468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:39.344637 26456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:39.344884 26456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:39.346064 26456 hybrid_clock.cc:648] HybridClock initialized: now 1786173279346051 us; error 44 us; skew 500 ppm
I20260808 07:14:39.347389 26456 webserver.cc:460] Webserver started at http://127.30.54.67:37201/ using document root <none> and password file <none>
I20260808 07:14:39.347636 26456 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:39.347709 26456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:39.347867 26456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:39.348835 26456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "b673c37f9df84402aa25060263291327"
format_stamp: "Formatted at 2026-08-08 07:14:39 on dist-test-slave-znh6"
server_key: "b7a4cd3e38052af4627b59fab4c154af"
server_key_iv: "209f8e855c975eab884eac721be9e9a9"
server_key_version: "encryptionkey@0"
I20260808 07:14:39.349210 26456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "b673c37f9df84402aa25060263291327"
format_stamp: "Formatted at 2026-08-08 07:14:39 on dist-test-slave-znh6"
server_key: "b7a4cd3e38052af4627b59fab4c154af"
server_key_iv: "209f8e855c975eab884eac721be9e9a9"
server_key_version: "encryptionkey@0"
I20260808 07:14:39.350448 26456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:14:39.351197 26475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.351370 26456 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.351469 26456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "b673c37f9df84402aa25060263291327"
format_stamp: "Formatted at 2026-08-08 07:14:39 on dist-test-slave-znh6"
server_key: "b7a4cd3e38052af4627b59fab4c154af"
server_key_iv: "209f8e855c975eab884eac721be9e9a9"
server_key_version: "encryptionkey@0"
I20260808 07:14:39.351548 26456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:39.367098 26456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:39.367363 26456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:39.367507 26456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:39.367799 26456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:39.368130 26456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:39.368191 26456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.368224 26456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:39.368252 26456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:39.374182 26456 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:41683
I20260808 07:14:39.374217 26640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:41683 every 8 connection(s)
I20260808 07:14:39.374562 26456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:14:39.379377 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 26456
I20260808 07:14:39.379467 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:14:39.379714 30937 external_mini_cluster.cc:1428] Setting key 9d8ee714122f00de485173d09eeb7e85
I20260808 07:14:39.379894 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.375359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:49145 (local address 127.30.54.126:34467)
0808 07:14:39.375447 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.375449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.375648 (+ 199us) server_negotiation.cc:407] Connection header received
0808 07:14:39.376146 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.376147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.376161 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.376224 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:39.376882 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.377399 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.378369 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.378577 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.378655 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.379577 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:39.379579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:39.379579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:39.379586 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:39.379603 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:39.379608 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:39.379687 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.379772 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.379798 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:14:39.380095 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.375305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34467 (local address 127.30.54.67:49145)
0808 07:14:39.375515 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:39.375555 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:39.376037 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:39.376297 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:39.376301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:39.376310 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:39.376785 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.376789 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.377498 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.377499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.378265 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.378267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.378663 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.378663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.378800 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.379355 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:14:39.379356 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:39.379382 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:14:39.379404 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:39.379689 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:39.379693 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:39.379696 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:39.379743 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:14:39.379865 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260808 07:14:39.380496 26641 heartbeater.cc:347] Connected to a master server at 127.30.54.126:34467
I20260808 07:14:39.380609 26641 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:39.380786 26641 heartbeater.cc:511] Master 127.30.54.126:34467 requested a full tablet report, sending...
I20260808 07:14:39.381155 25999 ts_manager.cc:194] Registered new tserver with Master: b673c37f9df84402aa25060263291327 (127.30.54.67:41683)
I20260808 07:14:39.381523 25999 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.67:49145
I20260808 07:14:39.391659 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:14:39.396121 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.393076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.30.54.126:34467)
0808 07:14:39.393140 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.393142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.393221 (+ 79us) server_negotiation.cc:407] Connection header received
0808 07:14:39.393307 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.393308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.393324 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.393389 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:39.393563 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.394025 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.394779 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.395023 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.395098 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.395827 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:39.395828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:39.395828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:39.395832 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:39.395846 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:39.395850 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:39.395946 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.396028 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.396038 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:14:39.397840 25999 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:14:39.399583 25999 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:14:39.405999 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.401835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:40891 (local address 127.0.0.1:45828)
0808 07:14:39.402130 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:39.402140 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:39.402196 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:39.402412 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:39.402415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:39.402418 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:39.402488 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.402489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.403594 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.403595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.404777 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.404780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.405706 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.405707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.405772 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.405798 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 07:14:39.405835 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 07:14:39.405891 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":3,"thread_start_us":90,"threads_started":1}
I20260808 07:14:39.406205 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.401925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.30.54.66:40891)
0808 07:14:39.402151 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.402156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.402190 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:14:39.402267 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.402268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.402285 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.402614 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:39.402680 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.403491 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.404872 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.405600 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.405971 (+ 371us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.406053 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.406078 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.406121 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":105,"threads_started":1}
I20260808 07:14:39.406193 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.401834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:41683 (local address 127.0.0.1:36918)
0808 07:14:39.402050 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:39.402085 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:39.402158 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:39.402565 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:39.402566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:39.402568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:39.402618 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.402620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.403969 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.403970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.405026 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.405029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.405984 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.405985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.406044 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.406047 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:39.406098 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:14:39.406130 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":27,"thread_start_us":97,"threads_started":1}
I20260808 07:14:39.406032 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.402379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.30.54.65:41229)
0808 07:14:39.402573 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.402577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.402586 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:39.402622 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.402624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.402638 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.402710 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:39.402953 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.404054 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.405114 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.405595 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.405683 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.405775 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.405903 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.405955 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
I20260808 07:14:39.406476 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.401909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36918 (local address 127.30.54.67:41683)
0808 07:14:39.402337 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.402341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.402352 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:14:39.402387 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.402389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.402404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.402497 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:39.402699 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.403875 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.405116 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.405810 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.405870 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.405947 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.406384 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.406419 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":72,"threads_started":1}
I20260808 07:14:39.406548 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.402108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:41229 (local address 127.0.0.1:55144)
0808 07:14:39.402381 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:39.402391 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:39.402433 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:39.402775 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:39.402776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:39.402778 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:39.402809 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.402861 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.404135 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.404136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.405028 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.405031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.405686 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.405688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.405768 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.405782 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:14:39.405873 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:14:39.405919 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":4,"thread_start_us":74,"threads_started":1}
I20260808 07:14:39.407003 26355 tablet_service.cc:1467] Processing CreateTablet for tablet 2aadd99fdccf45ac9237175e09c75acb (DEFAULT_TABLE table=test-table [id=8aae5bbe676a4a3895b64eff07bfbbd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:39.407171 26539 tablet_service.cc:1467] Processing CreateTablet for tablet 2aadd99fdccf45ac9237175e09c75acb (DEFAULT_TABLE table=test-table [id=8aae5bbe676a4a3895b64eff07bfbbd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:39.407413 26355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aadd99fdccf45ac9237175e09c75acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:39.407482 26539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aadd99fdccf45ac9237175e09c75acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:39.409883 26659 tablet_bootstrap.cc:492] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: Bootstrap starting.
I20260808 07:14:39.410082 26660 tablet_bootstrap.cc:492] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: Bootstrap starting.
I20260808 07:14:39.410353 26172 tablet_service.cc:1467] Processing CreateTablet for tablet 2aadd99fdccf45ac9237175e09c75acb (DEFAULT_TABLE table=test-table [id=8aae5bbe676a4a3895b64eff07bfbbd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:39.410703 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aadd99fdccf45ac9237175e09c75acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:39.410804 26660 tablet_bootstrap.cc:654] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:39.411166 26660 log.cc:824] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:39.411443 26659 tablet_bootstrap.cc:654] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:39.411762 26659 log.cc:824] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:39.412029 26660 tablet_bootstrap.cc:492] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: No bootstrap required, opened a new log
I20260808 07:14:39.412106 26660 ts_tablet_manager.cc:1397] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:14:39.412453 26659 tablet_bootstrap.cc:492] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: No bootstrap required, opened a new log
I20260808 07:14:39.412539 26659 ts_tablet_manager.cc:1397] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:14:39.414242 26664 tablet_bootstrap.cc:492] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7: Bootstrap starting.
I20260808 07:14:39.414661 26660 raft_consensus.cc:348] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } }
I20260808 07:14:39.414829 26660 raft_consensus.cc:374] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:39.414901 26660 raft_consensus.cc:729] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b673c37f9df84402aa25060263291327, State: Initialized, Role: FOLLOWER
I20260808 07:14:39.414947 26664 tablet_bootstrap.cc:654] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:39.414901 26659 raft_consensus.cc:348] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } }
I20260808 07:14:39.415024 26660 consensus_queue.cc:260] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } }
I20260808 07:14:39.415063 26659 raft_consensus.cc:374] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:39.415117 26659 raft_consensus.cc:729] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ab0750585064f16a793430500c504fb, State: Initialized, Role: FOLLOWER
I20260808 07:14:39.415256 26664 log.cc:824] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:39.415251 26659 consensus_queue.cc:260] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } }
I20260808 07:14:39.415287 26660 ts_tablet_manager.cc:1428] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:14:39.415438 26641 heartbeater.cc:503] Master 127.30.54.126:34467 was elected leader, sending a full tablet report...
I20260808 07:14:39.415838 26659 ts_tablet_manager.cc:1428] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:39.415874 26453 heartbeater.cc:503] Master 127.30.54.126:34467 was elected leader, sending a full tablet report...
I20260808 07:14:39.416034 26664 tablet_bootstrap.cc:492] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7: No bootstrap required, opened a new log
I20260808 07:14:39.416115 26664 ts_tablet_manager.cc:1397] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:14:39.418269 26664 raft_consensus.cc:348] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } }
I20260808 07:14:39.418381 26664 raft_consensus.cc:374] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:39.418401 26664 raft_consensus.cc:729] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4220d86ccb44df992ad15d9a91a4aa7, State: Initialized, Role: FOLLOWER
I20260808 07:14:39.418473 26664 consensus_queue.cc:260] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } }
I20260808 07:14:39.418644 26664 ts_tablet_manager.cc:1428] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 07:14:39.418754 26269 heartbeater.cc:503] Master 127.30.54.126:34467 was elected leader, sending a full tablet report...
W20260808 07:14:39.497012 26454 tablet.cc:2369] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:14:39.624006 26270 tablet.cc:2369] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:14:39.625113 26642 tablet.cc:2369] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:14:39.632078 26668 raft_consensus.cc:482] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:39.632210 26668 raft_consensus.cc:504] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } }
I20260808 07:14:39.632579 26668 leader_election.cc:288] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ab0750585064f16a793430500c504fb (127.30.54.66:40891), b673c37f9df84402aa25060263291327 (127.30.54.67:41683)
I20260808 07:14:39.635505 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.632857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:51953 (local address 127.30.54.67:41683)
0808 07:14:39.632933 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.632936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.632984 (+ 48us) server_negotiation.cc:407] Connection header received
0808 07:14:39.633069 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.633070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.633083 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.633147 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:39.633316 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.633787 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.634761 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.635140 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.635227 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.635236 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.635396 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.635432 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:14:39.635516 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.632822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:58693 (local address 127.30.54.66:40891)
0808 07:14:39.632876 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.632878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.632950 (+ 72us) server_negotiation.cc:407] Connection header received
0808 07:14:39.633010 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.633015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.633024 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.633089 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:39.633250 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.633735 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.634747 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.635161 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.635247 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.635256 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.635405 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.635443 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:14:39.635591 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.632746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:41683 (local address 127.30.54.65:51953)
0808 07:14:39.632925 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:39.632936 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:39.632999 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:39.633154 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:39.633154 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:39.633156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:39.633239 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.633240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.633876 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.633877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.634661 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.634664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.635228 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.635229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.635309 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.635313 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:39.635379 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:14:39.635422 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":62,"threads_started":1}
I20260808 07:14:39.635576 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.632734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:40891 (local address 127.30.54.65:58693)
0808 07:14:39.632856 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:39.632868 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:39.632939 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:39.633095 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:39.633095 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:39.633097 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:39.633178 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.633180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.633828 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.633830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.634652 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:39.634654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.635259 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.635260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:39.635327 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.635331 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:39.635400 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:14:39.635444 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":46,"thread_start_us":58,"threads_started":1}
I20260808 07:14:39.635994 26387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aadd99fdccf45ac9237175e09c75acb" candidate_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab0750585064f16a793430500c504fb" is_pre_election: true
I20260808 07:14:39.636004 26574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aadd99fdccf45ac9237175e09c75acb" candidate_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b673c37f9df84402aa25060263291327" is_pre_election: true
I20260808 07:14:39.636154 26387 raft_consensus.cc:2454] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4220d86ccb44df992ad15d9a91a4aa7 in term 0.
I20260808 07:14:39.636154 26574 raft_consensus.cc:2454] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4220d86ccb44df992ad15d9a91a4aa7 in term 0.
I20260808 07:14:39.636410 26104 leader_election.cc:302] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ab0750585064f16a793430500c504fb, f4220d86ccb44df992ad15d9a91a4aa7; no voters:
I20260808 07:14:39.636560 26668 raft_consensus.cc:2781] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:14:39.636634 26668 raft_consensus.cc:482] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:14:39.636655 26668 raft_consensus.cc:3037] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:39.637341 26668 raft_consensus.cc:504] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } }
I20260808 07:14:39.637476 26668 leader_election.cc:288] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [CANDIDATE]: Term 1 election: Requested vote from peers 6ab0750585064f16a793430500c504fb (127.30.54.66:40891), b673c37f9df84402aa25060263291327 (127.30.54.67:41683)
I20260808 07:14:39.637642 26574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aadd99fdccf45ac9237175e09c75acb" candidate_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b673c37f9df84402aa25060263291327"
I20260808 07:14:39.637642 26387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aadd99fdccf45ac9237175e09c75acb" candidate_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ab0750585064f16a793430500c504fb"
I20260808 07:14:39.637728 26387 raft_consensus.cc:3037] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:39.637728 26574 raft_consensus.cc:3037] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:39.638652 26574 raft_consensus.cc:2454] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4220d86ccb44df992ad15d9a91a4aa7 in term 1.
I20260808 07:14:39.638652 26387 raft_consensus.cc:2454] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4220d86ccb44df992ad15d9a91a4aa7 in term 1.
I20260808 07:14:39.638907 26102 leader_election.cc:302] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b673c37f9df84402aa25060263291327, f4220d86ccb44df992ad15d9a91a4aa7; no voters:
I20260808 07:14:39.639014 26668 raft_consensus.cc:2781] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:39.639174 26668 raft_consensus.cc:686] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 1 LEADER]: Becoming Leader. State: Replica: f4220d86ccb44df992ad15d9a91a4aa7, State: Running, Role: LEADER
I20260808 07:14:39.639269 26668 consensus_queue.cc:237] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } }
I20260808 07:14:39.640100 25999 catalog_manager.cc:5495] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4220d86ccb44df992ad15d9a91a4aa7 (127.30.54.65). New cstate: current_term: 1 leader_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4220d86ccb44df992ad15d9a91a4aa7" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 41229 } health_report { overall_health: HEALTHY } } }
I20260808 07:14:39.688297 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.685748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.30.54.65:41229)
0808 07:14:39.685817 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:39.685819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:39.685829 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:14:39.685872 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:39.685873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:39.685896 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:39.685997 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:14:39.686146 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.686705 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.687498 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:39.687708 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:39.687785 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:39.687946 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:14:39.688042 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:14:39.688152 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:39.688205 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 07:14:39.688216 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:14:39.690121 26574 raft_consensus.cc:1261] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [term 1 FOLLOWER]: Refusing update from remote peer f4220d86ccb44df992ad15d9a91a4aa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:39.690124 26387 raft_consensus.cc:1261] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [term 1 FOLLOWER]: Refusing update from remote peer f4220d86ccb44df992ad15d9a91a4aa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:39.690532 26668 consensus_queue.cc:1035] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b673c37f9df84402aa25060263291327" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 41683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:39.690691 26671 consensus_queue.cc:1035] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ab0750585064f16a793430500c504fb" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:39.693102 26674 mvcc.cc:205] Tried to move back new op lower bound from 7316165753608060928 to 7316165753403052032. Current Snapshot: MvccSnapshot[applied={T|T < 7316165753608060928}]
I20260808 07:14:39.693308 26676 mvcc.cc:205] Tried to move back new op lower bound from 7316165753608060928 to 7316165753403052032. Current Snapshot: MvccSnapshot[applied={T|T < 7316165753608060928}]
I20260808 07:14:39.693799 26675 mvcc.cc:205] Tried to move back new op lower bound from 7316165753608060928 to 7316165753403052032. Current Snapshot: MvccSnapshot[applied={T|T < 7316165753608060928}]
I20260808 07:14:39.696452 25997 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:39.696534 25997 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:39.697314 25997 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b59133238cf451bb4cc76e5f6a18f36: Sending DeleteTablet for 3 replicas of tablet 2aadd99fdccf45ac9237175e09c75acb
I20260808 07:14:39.697638 26355 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aadd99fdccf45ac9237175e09c75acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:39 UTC) from {username='slave'} at 127.0.0.1:45828
I20260808 07:14:39.697656 26172 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aadd99fdccf45ac9237175e09c75acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:39 UTC) from {username='slave'} at 127.0.0.1:55144
I20260808 07:14:39.697793 26539 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aadd99fdccf45ac9237175e09c75acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:39 UTC) from {username='slave'} at 127.0.0.1:36918
I20260808 07:14:39.697880 26690 tablet_replica.cc:330] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: stopping tablet replica
I20260808 07:14:39.697978 26690 raft_consensus.cc:2229] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:39.698079 26691 tablet_replica.cc:330] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: stopping tablet replica
I20260808 07:14:39.698165 26691 raft_consensus.cc:2229] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:39.698257 26691 raft_consensus.cc:2258] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:39.698354 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 26083
I20260808 07:14:39.698410 26689 tablet_replica.cc:330] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7: stopping tablet replica
I20260808 07:14:39.698499 26689 raft_consensus.cc:2229] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:14:39.698608 26689 raft_consensus.cc:2258] T 2aadd99fdccf45ac9237175e09c75acb P f4220d86ccb44df992ad15d9a91a4aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:39.704978 26691 ts_tablet_manager.cc:1905] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:39.698086 26690 raft_consensus.cc:2258] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:39.705790 26690 ts_tablet_manager.cc:1905] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:39.706912 26691 ts_tablet_manager.cc:1918] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:39.706988 26691 log.cc:1196] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/2aadd99fdccf45ac9237175e09c75acb
I20260808 07:14:39.707268 26691 ts_tablet_manager.cc:1939] T 2aadd99fdccf45ac9237175e09c75acb P b673c37f9df84402aa25060263291327: Deleting consensus metadata
I20260808 07:14:39.707549 26690 ts_tablet_manager.cc:1918] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:39.707625 26690 log.cc:1196] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/2aadd99fdccf45ac9237175e09c75acb
I20260808 07:14:39.707705 25978 catalog_manager.cc:4841] TS b673c37f9df84402aa25060263291327 (127.30.54.67:41683): tablet 2aadd99fdccf45ac9237175e09c75acb (table test-table [id=8aae5bbe676a4a3895b64eff07bfbbd6]) successfully deleted
I20260808 07:14:39.707875 26690 ts_tablet_manager.cc:1939] T 2aadd99fdccf45ac9237175e09c75acb P 6ab0750585064f16a793430500c504fb: Deleting consensus metadata
I20260808 07:14:39.708268 25980 catalog_manager.cc:4841] TS 6ab0750585064f16a793430500c504fb (127.30.54.66:40891): tablet 2aadd99fdccf45ac9237175e09c75acb (table test-table [id=8aae5bbe676a4a3895b64eff07bfbbd6]) successfully deleted
W20260808 07:14:39.708820 25979 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:41229 (error 108)
I20260808 07:14:39.709286 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:39.709168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:41229 (local address 127.0.0.1:55164)
0808 07:14:39.709223 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:39.709260 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:14:39.709398 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 26272
W20260808 07:14:39.709441 25979 catalog_manager.cc:4584] TS f4220d86ccb44df992ad15d9a91a4aa7 (127.30.54.65:41229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2aadd99fdccf45ac9237175e09c75acb: Network error: Client connection negotiation failed: client connection to 127.30.54.65:41229: connect: Connection refused (error 111)
I20260808 07:14:39.721812 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 26456
I20260808 07:14:39.729452 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 25954
2026-08-08T07:14:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (910 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:39 dist-test-slave-znh6 krb5kdc[26696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:39 dist-test-slave-znh6 krb5kdc[26696](info): set up 2 sockets
Aug 08 07:14:39 dist-test-slave-znh6 krb5kdc[26696](info): commencing operation
krb5kdc: starting...
W20260808 07:14:40.953421 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:41 dist-test-slave-znh6 krb5kdc[26696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:14:41.069435 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43751
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:43751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:41.148989 26714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:41.149183 26714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:41.149205 26714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:41.150261 26714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:41.150298 26714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:41.150312 26714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:41.150324 26714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:41.150337 26714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:41.151716 26714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:43751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:43751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:41.151906 26714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:41.152117 26714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:41.156960 26723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:41.157009 26721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:41.157105 26714 server_base.cc:1034] running on GCE node
W20260808 07:14:41.156994 26720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:41.157421 26714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:41.157691 26714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:41.158861 26714 hybrid_clock.cc:648] HybridClock initialized: now 1786173281158842 us; error 31 us; skew 500 ppm
Aug 08 07:14:41 dist-test-slave-znh6 krb5kdc[26696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173281, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:41.161265 26714 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:14:41.161679 26714 webserver.cc:460] Webserver started at http://127.30.54.126:34479/ using document root <none> and password file <none>
I20260808 07:14:41.161899 26714 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:41.161965 26714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:41.162104 26714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:41.163163 26714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "740f07dae18547d2bb38d1d211ed5fd5"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "a64e391dff2b4bdb367f2cd48e998f3f"
server_key_iv: "a1242547a775fac575f30c4d6421557a"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.163538 26714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "740f07dae18547d2bb38d1d211ed5fd5"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "a64e391dff2b4bdb367f2cd48e998f3f"
server_key_iv: "a1242547a775fac575f30c4d6421557a"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.164839 26714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:14:41.165524 26732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.165685 26714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:41.165782 26714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "740f07dae18547d2bb38d1d211ed5fd5"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "a64e391dff2b4bdb367f2cd48e998f3f"
server_key_iv: "a1242547a775fac575f30c4d6421557a"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.165863 26714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:41.180534 26714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:41.182143 26714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:41.182296 26714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:41.186266 26714 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:43751
I20260808 07:14:41.186309 26812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:43751 every 8 connection(s)
I20260808 07:14:41.186656 26714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:41.187477 26814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:41.189610 26814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5: Bootstrap starting.
I20260808 07:14:41.190191 26814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:41.190485 26814 log.cc:824] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:41.191220 26814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5: No bootstrap required, opened a new log
I20260808 07:14:41.192759 26814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740f07dae18547d2bb38d1d211ed5fd5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43751 } }
I20260808 07:14:41.192889 26814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:41.192911 26814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 740f07dae18547d2bb38d1d211ed5fd5, State: Initialized, Role: FOLLOWER
I20260808 07:14:41.193074 26814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740f07dae18547d2bb38d1d211ed5fd5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43751 } }
I20260808 07:14:41.193138 26814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:41.193159 26814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:41.193220 26814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:41.193758 26814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740f07dae18547d2bb38d1d211ed5fd5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43751 } }
I20260808 07:14:41.193799 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 26714
I20260808 07:14:41.193840 26814 leader_election.cc:302] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 740f07dae18547d2bb38d1d211ed5fd5; no voters:
I20260808 07:14:41.193953 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:41.194036 26814 leader_election.cc:288] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:41.194082 26817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:41.194168 30937 external_mini_cluster.cc:1428] Setting key 8c641337d50161f11c5506fea4b3a515
I20260808 07:14:41.194211 26817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [term 1 LEADER]: Becoming Leader. State: Replica: 740f07dae18547d2bb38d1d211ed5fd5, State: Running, Role: LEADER
I20260808 07:14:41.194298 26814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:41.194291 26817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740f07dae18547d2bb38d1d211ed5fd5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43751 } }
I20260808 07:14:41.195202 26820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "740f07dae18547d2bb38d1d211ed5fd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740f07dae18547d2bb38d1d211ed5fd5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43751 } } }
I20260808 07:14:41.195302 26820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:41.195438 26820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 740f07dae18547d2bb38d1d211ed5fd5. Latest consensus state: current_term: 1 leader_uuid: "740f07dae18547d2bb38d1d211ed5fd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740f07dae18547d2bb38d1d211ed5fd5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 43751 } } }
I20260808 07:14:41.195508 26820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:41.195600 26832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:41.196210 26832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:41.197825 26832 catalog_manager.cc:1347] Generated new cluster ID: 6b119b05e4fc4bf885b512911dff4850
I20260808 07:14:41.197881 26832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:14:41 dist-test-slave-znh6 krb5kdc[26696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:41.206409 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.194640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.30.54.126:43751)
0808 07:14:41.195030 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.195040 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.195074 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:14:41.195188 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.195196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.195248 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.196244 (+ 996us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:41.196339 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.197692 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.198766 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.199421 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.199515 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.202225 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:41.202229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:41.202229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:41.202241 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:41.204839 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.205199 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.205202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.205202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.205322 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.205519 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.205520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.205521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.205819 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:41.205970 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.206055 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.206191 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":78,"threads_started":1}
I20260808 07:14:41.221369 26832 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:41.221843 26832 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:41.229194 26832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5: Generated new TSK 0
I20260808 07:14:41.229372 26832 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.54.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
Entry for principal kudu/127.30.54.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab.
I20260808 07:14:41.261323 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43751
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:41.341454 26851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:41.341642 26851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:41.341666 26851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:41.342792 26851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:41.342857 26851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:41.342886 26851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:14:41.344321 26851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.65.keytab
--principal=kudu/127.30.54.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:41.344573 26851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:41.344787 26851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:14:41.349875 26851 server_base.cc:1034] running on GCE node
W20260808 07:14:41.349839 26860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:41.349820 26856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:41.349828 26857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:41.350281 26851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:41.350518 26851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:41.351660 26851 hybrid_clock.cc:648] HybridClock initialized: now 1786173281351640 us; error 35 us; skew 500 ppm
Aug 08 07:14:41 dist-test-slave-znh6 krb5kdc[26696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173281, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:41.354249 26851 init.cc:377] Logged in from keytab as kudu/127.30.54.65@KRBTEST.COM (short username kudu)
I20260808 07:14:41.354666 26851 webserver.cc:460] Webserver started at http://127.30.54.65:42377/ using document root <none> and password file <none>
I20260808 07:14:41.354936 26851 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:41.355005 26851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:41.355156 26851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:41.356201 26851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "150d7fec4889449aa0ce76408e2d361d"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "2ca75d8741705cfa511b62a73abf451b"
server_key_iv: "cf2fe4468c1d048c7859526d7a2543f0"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.356567 26851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "150d7fec4889449aa0ce76408e2d361d"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "2ca75d8741705cfa511b62a73abf451b"
server_key_iv: "cf2fe4468c1d048c7859526d7a2543f0"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.357892 26851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:14:41.358613 26867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.358783 26851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:41.358911 26851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "150d7fec4889449aa0ce76408e2d361d"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "2ca75d8741705cfa511b62a73abf451b"
server_key_iv: "cf2fe4468c1d048c7859526d7a2543f0"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.358989 26851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:41.393915 26851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:41.396397 26851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:41.396560 26851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:41.396816 26851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:41.397141 26851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:41.397194 26851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.397230 26851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:41.397274 26851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.403216 26851 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:35807
I20260808 07:14:41.403280 27027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:35807 every 8 connection(s)
I20260808 07:14:41.403584 26851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:41.406363 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 26851
I20260808 07:14:41.406509 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:14:41.406706 30937 external_mini_cluster.cc:1428] Setting key 068d77ad6b5a76d07b31488d10956f31
Aug 08 07:14:41 dist-test-slave-znh6 krb5kdc[26696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173281, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.65@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:41.413477 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.404274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:34073 (local address 127.30.54.126:43751)
0808 07:14:41.404320 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.404322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.404569 (+ 247us) server_negotiation.cc:407] Connection header received
0808 07:14:41.405100 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.405101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.405117 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.405168 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:41.406114 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.406702 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.407889 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.408095 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.408178 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.410562 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:41.410564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:41.410564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:41.410570 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:41.411987 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.412492 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.412493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.412493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.412555 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.412847 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.412847 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.412847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.412954 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:41.413003 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.413350 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.413415 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:14:41.413823 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.404261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43751 (local address 127.30.54.65:34073)
0808 07:14:41.404450 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.404485 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:41.404989 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:41.405239 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:41.405243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:41.405818 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:41.406013 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.406019 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.406794 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.406795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.407794 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.407797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.408192 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.408193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.408323 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.408799 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:41.408807 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:41.410475 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:41.412065 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:41.412073 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:41.412076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:41.412403 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:41.412624 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:41.412625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:41.412626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:41.412780 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:41.413018 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:41.413025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:41.413202 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 07:14:41.413336 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 07:14:41.413550 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260808 07:14:41.415314 27028 heartbeater.cc:347] Connected to a master server at 127.30.54.126:43751
I20260808 07:14:41.415433 27028 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:41.415618 27028 heartbeater.cc:511] Master 127.30.54.126:43751 requested a full tablet report, sending...
I20260808 07:14:41.416230 26761 ts_manager.cc:194] Registered new tserver with Master: 150d7fec4889449aa0ce76408e2d361d (127.30.54.65:35807)
WARNING: no policy specified for kudu/127.30.54.66@KRBTEST.COM; defaulting to no policy
I20260808 07:14:41.417095 26761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.65@KRBTEST.COM'} at 127.30.54.65:34073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
Entry for principal kudu/127.30.54.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab.
I20260808 07:14:41.433701 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43751
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:41.513406 27035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:41.513589 27035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:41.513612 27035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:41.514637 27035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:41.514670 27035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:41.514695 27035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:14:41.516083 27035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.66.keytab
--principal=kudu/127.30.54.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:41.516259 27035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:41.516422 27035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:41.521221 27046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:41.521216 27042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:41.521219 27044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:41.521370 27035 server_base.cc:1034] running on GCE node
I20260808 07:14:41.521636 27035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:41.521891 27035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:41.523037 27035 hybrid_clock.cc:648] HybridClock initialized: now 1786173281523028 us; error 30 us; skew 500 ppm
Aug 08 07:14:41 dist-test-slave-znh6 krb5kdc[26696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173281, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:41.525424 27035 init.cc:377] Logged in from keytab as kudu/127.30.54.66@KRBTEST.COM (short username kudu)
I20260808 07:14:41.525848 27035 webserver.cc:460] Webserver started at http://127.30.54.66:40761/ using document root <none> and password file <none>
I20260808 07:14:41.526031 27035 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:41.526069 27035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:41.526160 27035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:41.527124 27035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "cf8eaed3fc6745659b5569ded4fa33f9"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "594f35a6d20369389d995c7576990502"
server_key_iv: "03f46964ffbc6573f583bdde99113fa1"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.527420 27035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "cf8eaed3fc6745659b5569ded4fa33f9"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "594f35a6d20369389d995c7576990502"
server_key_iv: "03f46964ffbc6573f583bdde99113fa1"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.528594 27035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:14:41.529237 27059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.529415 27035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:41.529497 27035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "cf8eaed3fc6745659b5569ded4fa33f9"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "594f35a6d20369389d995c7576990502"
server_key_iv: "03f46964ffbc6573f583bdde99113fa1"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.529567 27035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:41.547467 27035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:41.549641 27035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:41.549799 27035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:41.550004 27035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:41.550299 27035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:41.550351 27035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.550387 27035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:41.550428 27035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.556555 27035 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:40961
I20260808 07:14:41.556602 27225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:40961 every 8 connection(s)
I20260808 07:14:41.556922 27035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:14:41.558319 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 27035
I20260808 07:14:41.558449 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:14:41.558634 30937 external_mini_cluster.cc:1428] Setting key 73651f8cf8294312b7b3765f5cb32f28
Aug 08 07:14:41 dist-test-slave-znh6 krb5kdc[26696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173281, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.66@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:41.567253 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.557648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:56489 (local address 127.30.54.126:43751)
0808 07:14:41.557700 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.557703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.557890 (+ 187us) server_negotiation.cc:407] Connection header received
0808 07:14:41.558330 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.558332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.558345 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.558455 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:41.559569 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.560305 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.561346 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.561598 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.561695 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.564158 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:41.564159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:41.564160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:41.564167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:41.565704 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.566222 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.566223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.566223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.566278 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.566589 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.566589 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.566590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.566707 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:41.566767 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.567110 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.567182 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:14:41.567512 27228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.557626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43751 (local address 127.30.54.66:56489)
0808 07:14:41.557780 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.557812 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:41.558242 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:41.558537 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:41.558540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:41.559289 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:41.559480 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.559484 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.560407 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.560408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.561222 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.561225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.561702 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.561703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.561868 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.562360 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:41.562373 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:41.564045 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:41.565793 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:41.565801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:41.565803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:41.566141 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:41.566354 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:41.566355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:41.566356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:41.566515 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:41.566779 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:41.566787 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:41.566962 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 07:14:41.567073 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:14:41.567252 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260808 07:14:41.568030 27226 heartbeater.cc:347] Connected to a master server at 127.30.54.126:43751
I20260808 07:14:41.568130 27226 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:41.568331 27226 heartbeater.cc:511] Master 127.30.54.126:43751 requested a full tablet report, sending...
I20260808 07:14:41.568737 26761 ts_manager.cc:194] Registered new tserver with Master: cf8eaed3fc6745659b5569ded4fa33f9 (127.30.54.66:40961)
I20260808 07:14:41.569133 26761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.66@KRBTEST.COM'} at 127.30.54.66:56489
WARNING: no policy specified for kudu/127.30.54.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
Entry for principal kudu/127.30.54.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab.
I20260808 07:14:41.586915 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43751
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:41.668550 27232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:41.668745 27232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:41.668767 27232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:41.669885 27232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:41.669929 27232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:41.669955 27232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:14:41.671444 27232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.67.keytab
--principal=kudu/127.30.54.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:43751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:41.671628 27232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:41.671797 27232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:41.676867 27243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:41.676904 27241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:41.676878 27239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:41.677258 27232 server_base.cc:1034] running on GCE node
I20260808 07:14:41.677464 27232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:41.677687 27232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:41.678880 27232 hybrid_clock.cc:648] HybridClock initialized: now 1786173281678877 us; error 47 us; skew 500 ppm
Aug 08 07:14:41 dist-test-slave-znh6 krb5kdc[26696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173281, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:41.681295 27232 init.cc:377] Logged in from keytab as kudu/127.30.54.67@KRBTEST.COM (short username kudu)
I20260808 07:14:41.681775 27232 webserver.cc:460] Webserver started at http://127.30.54.67:45515/ using document root <none> and password file <none>
I20260808 07:14:41.682008 27232 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:41.682071 27232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:41.682204 27232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:41.683331 27232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "a24ec7167c02431383afff7066b2cc8a"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "194180e1ae46df05c11020117818f774"
server_key_iv: "da84566c30fea946a73f200af8952cd9"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.683710 27232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "a24ec7167c02431383afff7066b2cc8a"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "194180e1ae46df05c11020117818f774"
server_key_iv: "da84566c30fea946a73f200af8952cd9"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.685010 27232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:14:41.685753 27252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.685938 27232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:41.686033 27232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "a24ec7167c02431383afff7066b2cc8a"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "194180e1ae46df05c11020117818f774"
server_key_iv: "da84566c30fea946a73f200af8952cd9"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.686112 27232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:41.701287 27232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:41.703044 27232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:41.703187 27232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:41.703402 27232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:41.703696 27232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:41.703745 27232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.703786 27232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:41.703815 27232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.709724 27232 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:33689
I20260808 07:14:41.709772 27402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:33689 every 8 connection(s)
I20260808 07:14:41.710057 27232 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:14:41.711875 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 27232
I20260808 07:14:41.711982 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:14:41.712153 30937 external_mini_cluster.cc:1428] Setting key 336baacb846cf52feb3a0a3b5232dd5e
Aug 08 07:14:41 dist-test-slave-znh6 krb5kdc[26696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173281, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.67@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:14:41.719240 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.710708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:59775 (local address 127.30.54.126:43751)
0808 07:14:41.710762 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.710764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.710986 (+ 222us) server_negotiation.cc:407] Connection header received
0808 07:14:41.711496 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.711497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.711509 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.711570 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:41.712674 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.713163 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.714125 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.714335 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.714407 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.716524 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:41.716526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:41.716527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:41.716535 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:41.717825 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.718228 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.718229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.718230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.718295 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.718538 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.718539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.718539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.718656 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:41.718726 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.719068 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.719150 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:14:41.719419 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.710679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:43751 (local address 127.30.54.67:59775)
0808 07:14:41.710866 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.710909 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:41.711392 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:41.711639 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:41.711643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:41.712326 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:41.712593 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.712599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.713253 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.713254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.714037 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.714039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.714425 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.714426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.714548 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.715030 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:14:41.715038 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:41.716410 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:41.717908 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:41.717915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:41.717918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:41.718137 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:41.718368 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:14:41.718369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:14:41.718369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:14:41.718472 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:14:41.718729 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:41.718736 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:41.718937 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 07:14:41.719042 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:14:41.719193 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":82,"threads_started":1}
I20260808 07:14:41.719906 27403 heartbeater.cc:347] Connected to a master server at 127.30.54.126:43751
I20260808 07:14:41.719998 27403 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:41.720162 27403 heartbeater.cc:511] Master 127.30.54.126:43751 requested a full tablet report, sending...
I20260808 07:14:41.720520 26761 ts_manager.cc:194] Registered new tserver with Master: a24ec7167c02431383afff7066b2cc8a (127.30.54.67:33689)
I20260808 07:14:41.720890 26761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.54.67@KRBTEST.COM'} at 127.30.54.67:59775
I20260808 07:14:41.723716 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:14:41.730561 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.724989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.30.54.126:43751)
0808 07:14:41.725037 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.725038 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.725133 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:14:41.725186 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.725186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.725194 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.725247 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:41.725746 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.726217 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.726906 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.727088 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.727143 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.728194 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:41.728195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:41.728195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:41.728201 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:41.729590 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.729925 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.729926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.729926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.729972 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:41.730126 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:41.730127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:41.730127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:41.730245 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:41.730339 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.730455 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.730508 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:14:41.732018 26761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:14:41.733603 26761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:14:41.739914 27418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.735814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:40961 (local address 127.0.0.1:53392)
0808 07:14:41.736112 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.736122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:41.736161 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:41.736594 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:41.736595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:41.736597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:41.736638 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.736640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.737903 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.737904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.738652 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.738655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.739602 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.739603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.739691 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.739705 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:14:41.739776 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:14:41.739829 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":83,"threads_started":1}
I20260808 07:14:41.739943 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.735933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.30.54.66:40961)
0808 07:14:41.736416 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.736420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.736430 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:14:41.736468 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.736470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.736485 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.736711 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:41.736800 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.737815 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.738745 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.739507 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.739590 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.739650 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.739804 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.739851 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":98,"threads_started":1}
I20260808 07:14:41.740056 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.735884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33844 (local address 127.30.54.65:35807)
0808 07:14:41.736080 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.736084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.736100 (+ 16us) server_negotiation.cc:407] Connection header received
0808 07:14:41.736142 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.736145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.736161 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.736256 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:41.736450 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.737569 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.738884 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.739582 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.739643 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.739688 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.739925 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.739969 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":80,"threads_started":1}
I20260808 07:14:41.740088 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.736169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.30.54.67:33689)
0808 07:14:41.736441 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.736446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.736455 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:41.736489 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.736490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.736505 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.736738 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:41.736796 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.737403 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.739103 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.739614 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.739688 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.739741 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.739959 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.740001 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":92,"threads_started":1}
I20260808 07:14:41.740274 27420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.735741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:35807 (local address 127.0.0.1:33844)
0808 07:14:41.735958 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.735995 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:41.736060 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:41.736269 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:41.736273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:41.736275 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:41.736344 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.736346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.737675 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.737675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.738764 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.738767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.739731 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.739732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.739824 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.739829 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:14:41.740129 (+ 300us) client_negotiation.cc:240] Negotiation successful
0808 07:14:41.740179 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":66,"threads_started":1}
I20260808 07:14:41.740285 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.735998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:33689 (local address 127.0.0.1:52308)
0808 07:14:41.736260 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.736269 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:41.736305 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:41.736605 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:41.736606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:41.736608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:41.736662 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.736664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.737507 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.737508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.739023 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.739025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.739776 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.739776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.739834 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.739837 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:41.739884 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:14:41.739917 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":42,"thread_start_us":66,"threads_started":1}
I20260808 07:14:41.741107 27312 tablet_service.cc:1467] Processing CreateTablet for tablet 6c52c19c2419493b969e1b23daf4a145 (DEFAULT_TABLE table=test-table [id=b75c847f50ce4798bb4b724643b4fa2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:41.741107 26935 tablet_service.cc:1467] Processing CreateTablet for tablet 6c52c19c2419493b969e1b23daf4a145 (DEFAULT_TABLE table=test-table [id=b75c847f50ce4798bb4b724643b4fa2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:41.741421 27312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c52c19c2419493b969e1b23daf4a145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:41.741434 26935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c52c19c2419493b969e1b23daf4a145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:41.743417 27130 tablet_service.cc:1467] Processing CreateTablet for tablet 6c52c19c2419493b969e1b23daf4a145 (DEFAULT_TABLE table=test-table [id=b75c847f50ce4798bb4b724643b4fa2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:41.743746 27130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c52c19c2419493b969e1b23daf4a145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:41.744438 27431 tablet_bootstrap.cc:492] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d: Bootstrap starting.
I20260808 07:14:41.744971 27431 tablet_bootstrap.cc:654] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:41.745402 27431 log.cc:824] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:41.746033 27431 tablet_bootstrap.cc:492] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d: No bootstrap required, opened a new log
I20260808 07:14:41.746078 27431 ts_tablet_manager.cc:1397] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:14:41.746232 27433 tablet_bootstrap.cc:492] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9: Bootstrap starting.
I20260808 07:14:41.746467 27430 tablet_bootstrap.cc:492] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: Bootstrap starting.
I20260808 07:14:41.747079 27433 tablet_bootstrap.cc:654] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:41.747079 27430 tablet_bootstrap.cc:654] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:41.747675 27433 log.cc:824] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:41.747716 27430 log.cc:824] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:41.748116 27431 raft_consensus.cc:348] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } }
I20260808 07:14:41.748330 27431 raft_consensus.cc:374] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:41.748390 27431 raft_consensus.cc:729] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150d7fec4889449aa0ce76408e2d361d, State: Initialized, Role: FOLLOWER
I20260808 07:14:41.748437 27430 tablet_bootstrap.cc:492] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: No bootstrap required, opened a new log
I20260808 07:14:41.748512 27430 ts_tablet_manager.cc:1397] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:14:41.748518 27431 consensus_queue.cc:260] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } }
I20260808 07:14:41.748760 27433 tablet_bootstrap.cc:492] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9: No bootstrap required, opened a new log
I20260808 07:14:41.748831 27433 ts_tablet_manager.cc:1397] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:14:41.748811 27431 ts_tablet_manager.cc:1428] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:41.748826 27028 heartbeater.cc:503] Master 127.30.54.126:43751 was elected leader, sending a full tablet report...
I20260808 07:14:41.750739 27430 raft_consensus.cc:348] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } }
I20260808 07:14:41.750739 27433 raft_consensus.cc:348] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } }
I20260808 07:14:41.750936 27430 raft_consensus.cc:374] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:41.750936 27433 raft_consensus.cc:374] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:41.750978 27433 raft_consensus.cc:729] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8eaed3fc6745659b5569ded4fa33f9, State: Initialized, Role: FOLLOWER
I20260808 07:14:41.750978 27430 raft_consensus.cc:729] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24ec7167c02431383afff7066b2cc8a, State: Initialized, Role: FOLLOWER
I20260808 07:14:41.751091 27433 consensus_queue.cc:260] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } }
I20260808 07:14:41.751091 27430 consensus_queue.cc:260] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } }
I20260808 07:14:41.751353 27433 ts_tablet_manager.cc:1428] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:14:41.751477 27226 heartbeater.cc:503] Master 127.30.54.126:43751 was elected leader, sending a full tablet report...
I20260808 07:14:41.751515 27430 ts_tablet_manager.cc:1428] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:14:41.751578 27403 heartbeater.cc:503] Master 127.30.54.126:43751 was elected leader, sending a full tablet report...
I20260808 07:14:41.753331 27440 raft_consensus.cc:482] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:41.753394 27440 raft_consensus.cc:504] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } }
I20260808 07:14:41.753679 27440 leader_election.cc:288] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 150d7fec4889449aa0ce76408e2d361d (127.30.54.65:35807), cf8eaed3fc6745659b5569ded4fa33f9 (127.30.54.66:40961)
I20260808 07:14:41.756466 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.753917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:50585 (local address 127.30.54.65:35807)
0808 07:14:41.753967 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.753969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.754051 (+ 82us) server_negotiation.cc:407] Connection header received
0808 07:14:41.754113 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.754114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.754123 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.754177 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:41.754317 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.754751 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.755770 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.756146 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.756225 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.756233 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.756373 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.756405 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:14:41.756475 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.753864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:56247 (local address 127.30.54.66:40961)
0808 07:14:41.753980 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.753982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.754027 (+ 45us) server_negotiation.cc:407] Connection header received
0808 07:14:41.754099 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.754100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.754109 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.754154 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:41.754289 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.754750 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.755771 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.756146 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.756228 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.756235 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.756382 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.756413 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 07:14:41.756505 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.753812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:40961 (local address 127.30.54.67:56247)
0808 07:14:41.753984 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.753994 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:41.754041 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:41.754154 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:41.754154 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:41.754156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:41.754218 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.754219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.754857 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.754858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.755686 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.755688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.756238 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.756239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.756310 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.756313 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:41.756377 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:14:41.756414 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":44,"thread_start_us":97,"threads_started":1}
I20260808 07:14:41.756806 27161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c52c19c2419493b969e1b23daf4a145" candidate_uuid: "a24ec7167c02431383afff7066b2cc8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" is_pre_election: true
I20260808 07:14:41.756505 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.753812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:35807 (local address 127.30.54.67:50585)
0808 07:14:41.753983 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.753993 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:41.754053 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:41.754180 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:41.754180 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:41.754182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:41.754241 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.754243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.754843 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.754844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.755681 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:41.755684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.756224 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.756225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:41.756304 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.756308 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:41.756369 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:14:41.756406 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":32}
I20260808 07:14:41.756940 27161 raft_consensus.cc:2454] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a24ec7167c02431383afff7066b2cc8a in term 0.
I20260808 07:14:41.757148 27255 leader_election.cc:302] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24ec7167c02431383afff7066b2cc8a, cf8eaed3fc6745659b5569ded4fa33f9; no voters:
I20260808 07:14:41.757192 26965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c52c19c2419493b969e1b23daf4a145" candidate_uuid: "a24ec7167c02431383afff7066b2cc8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "150d7fec4889449aa0ce76408e2d361d" is_pre_election: true
I20260808 07:14:41.757311 26965 raft_consensus.cc:2454] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a24ec7167c02431383afff7066b2cc8a in term 0.
I20260808 07:14:41.757332 27440 raft_consensus.cc:2781] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:14:41.757369 27440 raft_consensus.cc:482] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:14:41.757385 27440 raft_consensus.cc:3037] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:41.757995 27440 raft_consensus.cc:504] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } }
I20260808 07:14:41.758113 27440 leader_election.cc:288] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [CANDIDATE]: Term 1 election: Requested vote from peers 150d7fec4889449aa0ce76408e2d361d (127.30.54.65:35807), cf8eaed3fc6745659b5569ded4fa33f9 (127.30.54.66:40961)
I20260808 07:14:41.758232 26965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c52c19c2419493b969e1b23daf4a145" candidate_uuid: "a24ec7167c02431383afff7066b2cc8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "150d7fec4889449aa0ce76408e2d361d"
I20260808 07:14:41.758279 27161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c52c19c2419493b969e1b23daf4a145" candidate_uuid: "a24ec7167c02431383afff7066b2cc8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8eaed3fc6745659b5569ded4fa33f9"
I20260808 07:14:41.758354 27161 raft_consensus.cc:3037] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:41.758350 26965 raft_consensus.cc:3037] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:41.759022 27161 raft_consensus.cc:2454] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24ec7167c02431383afff7066b2cc8a in term 1.
I20260808 07:14:41.759119 26965 raft_consensus.cc:2454] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24ec7167c02431383afff7066b2cc8a in term 1.
I20260808 07:14:41.759172 27255 leader_election.cc:302] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24ec7167c02431383afff7066b2cc8a, cf8eaed3fc6745659b5569ded4fa33f9; no voters:
I20260808 07:14:41.759274 27440 raft_consensus.cc:2781] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:41.759398 27440 raft_consensus.cc:686] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 1 LEADER]: Becoming Leader. State: Replica: a24ec7167c02431383afff7066b2cc8a, State: Running, Role: LEADER
I20260808 07:14:41.759467 27440 consensus_queue.cc:237] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } }
I20260808 07:14:41.760123 26760 catalog_manager.cc:5495] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to a24ec7167c02431383afff7066b2cc8a (127.30.54.67). New cstate: current_term: 1 leader_uuid: "a24ec7167c02431383afff7066b2cc8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a24ec7167c02431383afff7066b2cc8a" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 33689 } health_report { overall_health: HEALTHY } } }
I20260808 07:14:41.767091 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.764996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.30.54.67:33689)
0808 07:14:41.765052 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.765053 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.765061 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:14:41.765086 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.765086 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.765100 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.765161 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:14:41.765276 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.765712 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.766410 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.766590 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.766651 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.766776 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:14:41.766917 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:14:41.766992 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.767015 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.767032 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:14:41.768559 27161 raft_consensus.cc:1261] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [term 1 FOLLOWER]: Refusing update from remote peer a24ec7167c02431383afff7066b2cc8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:41.768563 26965 raft_consensus.cc:1261] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [term 1 FOLLOWER]: Refusing update from remote peer a24ec7167c02431383afff7066b2cc8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:41.768857 27442 consensus_queue.cc:1035] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf8eaed3fc6745659b5569ded4fa33f9" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 40961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:41.768970 27440 consensus_queue.cc:1035] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "150d7fec4889449aa0ce76408e2d361d" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 35807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:41.773783 26761 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:41.773859 26761 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:41.774613 26761 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 740f07dae18547d2bb38d1d211ed5fd5: Sending DeleteTablet for 3 replicas of tablet 6c52c19c2419493b969e1b23daf4a145
I20260808 07:14:41.774960 27130 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c52c19c2419493b969e1b23daf4a145 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:41 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:53392
I20260808 07:14:41.774972 26935 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c52c19c2419493b969e1b23daf4a145 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:41 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:33844
I20260808 07:14:41.775207 27459 tablet_replica.cc:330] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d: stopping tablet replica
I20260808 07:14:41.775272 27312 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c52c19c2419493b969e1b23daf4a145 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:41 UTC) from {username='kudu', principal='kudu/127.30.54.126@KRBTEST.COM'} at 127.0.0.1:52308
I20260808 07:14:41.775312 27459 raft_consensus.cc:2229] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:41.775411 27459 raft_consensus.cc:2258] T 6c52c19c2419493b969e1b23daf4a145 P 150d7fec4889449aa0ce76408e2d361d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:41.775475 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 26851
I20260808 07:14:41.777959 27460 tablet_replica.cc:330] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: stopping tablet replica
I20260808 07:14:41.778055 27460 raft_consensus.cc:2229] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:14:41.778172 27460 raft_consensus.cc:2258] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:41.778568 27460 ts_tablet_manager.cc:1905] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:41.780164 27460 ts_tablet_manager.cc:1918] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:41.780243 27460 log.cc:1196] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/6c52c19c2419493b969e1b23daf4a145
I20260808 07:14:41.780545 27460 ts_tablet_manager.cc:1939] T 6c52c19c2419493b969e1b23daf4a145 P a24ec7167c02431383afff7066b2cc8a: Deleting consensus metadata
I20260808 07:14:41.780925 26737 catalog_manager.cc:4841] TS a24ec7167c02431383afff7066b2cc8a (127.30.54.67:33689): tablet 6c52c19c2419493b969e1b23daf4a145 (table test-table [id=b75c847f50ce4798bb4b724643b4fa2d]) successfully deleted
W20260808 07:14:41.784228 26739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:35807 (error 108)
I20260808 07:14:41.784319 27458 tablet_replica.cc:330] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9: stopping tablet replica
I20260808 07:14:41.784433 27458 raft_consensus.cc:2229] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:41.784539 27458 raft_consensus.cc:2258] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:41.784811 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.784695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:35807 (local address 127.0.0.1:33846)
0808 07:14:41.784740 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.784771 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:35807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 07:14:41.784850 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 27035
I20260808 07:14:41.785000 27458 ts_tablet_manager.cc:1905] T 6c52c19c2419493b969e1b23daf4a145 P cf8eaed3fc6745659b5569ded4fa33f9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:14:41.787456 26739 catalog_manager.cc:4584] TS 150d7fec4889449aa0ce76408e2d361d (127.30.54.65:35807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c52c19c2419493b969e1b23daf4a145: Network error: Client connection negotiation failed: client connection to 127.30.54.65:35807: connect: Connection refused (error 111)
W20260808 07:14:41.791312 27428 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:14:41.791114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:40961 (local address 127.0.0.1:53406)
0808 07:14:41.791162 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:41.791174 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:41.791220 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:41.791262 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.66:40961: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 07:14:41.791570 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 27232
I20260808 07:14:41.802273 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 26714
2026-08-08T07:14:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2073 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T07:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:41Z Disabled control of system clock
I20260808 07:14:41.834640 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34813
--webserver_interface=127.30.54.126
--webserver_port=0
--builtin_ntp_servers=127.30.54.84:45591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:34813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:14:41.917578 27467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:41.917788 27467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:41.917809 27467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:41.919008 27467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:41.919046 27467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:41.919059 27467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:41.919072 27467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:41.919117 27467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:41.920415 27467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:45591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:34813
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:34813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:41.920652 27467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:41.920939 27467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:41.925652 27475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:41.925686 27467 server_base.cc:1034] running on GCE node
W20260808 07:14:41.925657 27478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:41.925673 27474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:41.926076 27467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:41.926353 27467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:41.927508 27467 hybrid_clock.cc:648] HybridClock initialized: now 1786173281927484 us; error 43 us; skew 500 ppm
I20260808 07:14:41.928715 27467 webserver.cc:460] Webserver started at http://127.30.54.126:39505/ using document root <none> and password file <none>
I20260808 07:14:41.928967 27467 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:41.929035 27467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:41.929174 27467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:41.930213 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "a31a745619e1a59347d5fbce66da029a"
server_key_iv: "cb21cab63025370398badee76d613d40"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.930564 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "a31a745619e1a59347d5fbce66da029a"
server_key_iv: "cb21cab63025370398badee76d613d40"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.931895 27467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:14:41.932587 27487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:41.932752 27467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:14:41.932837 27467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb"
format_stamp: "Formatted at 2026-08-08 07:14:41 on dist-test-slave-znh6"
server_key: "a31a745619e1a59347d5fbce66da029a"
server_key_iv: "cb21cab63025370398badee76d613d40"
server_key_version: "encryptionkey@0"
I20260808 07:14:41.932914 27467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:41.973502 27467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:41.973853 27467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:41.974027 27467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:41.978009 27467 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:34813
I20260808 07:14:41.978051 27561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:34813 every 8 connection(s)
I20260808 07:14:41.978417 27467 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:41.979534 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 27467
I20260808 07:14:41.979656 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:41.979619 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:41.979861 30937 external_mini_cluster.cc:1428] Setting key 89305e7c33cb8fb96dffd1e44cf028b0
I20260808 07:14:41.983119 27562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb: Bootstrap starting.
I20260808 07:14:41.983712 27562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:41.984053 27562 log.cc:824] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:41.984815 27562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb: No bootstrap required, opened a new log
I20260808 07:14:41.985955 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:41.980404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35080 (local address 127.30.54.126:34813)
0808 07:14:41.980674 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:41.980683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:41.980716 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:14:41.981238 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:41.981249 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:41.981368 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:41.981522 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:41.981745 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.983117 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.983922 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:41.984457 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:41.984590 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:41.985257 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:41.985261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:41.985262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:41.985472 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:41.985515 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:41.985530 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:41.985601 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:41.985660 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 07:14:41.985735 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":126,"threads_started":1}
I20260808 07:14:41.987149 27562 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34813 } }
I20260808 07:14:41.987275 27562 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:41.987300 27562 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc19b41fb5a4b6292fec00d4a41a6fb, State: Initialized, Role: FOLLOWER
I20260808 07:14:41.987409 27562 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34813 } }
I20260808 07:14:41.987483 27562 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:41.987504 27562 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:41.987530 27562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:41.988049 27562 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34813 } }
I20260808 07:14:41.988131 27562 leader_election.cc:302] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcc19b41fb5a4b6292fec00d4a41a6fb; no voters:
I20260808 07:14:41.988256 27562 leader_election.cc:288] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:41.988332 27568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:41.988482 27568 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [term 1 LEADER]: Becoming Leader. State: Replica: dcc19b41fb5a4b6292fec00d4a41a6fb, State: Running, Role: LEADER
I20260808 07:14:41.988550 27562 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:41.988610 27568 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34813 } }
I20260808 07:14:41.989136 27569 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34813 } } }
I20260808 07:14:41.989226 27569 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:41.989451 27583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:41.989661 27570 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcc19b41fb5a4b6292fec00d4a41a6fb. Latest consensus state: current_term: 1 leader_uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc19b41fb5a4b6292fec00d4a41a6fb" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 34813 } } }
I20260808 07:14:41.989814 27570 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:41.989954 27583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:41.991369 27583 catalog_manager.cc:1347] Generated new cluster ID: 8ac397d27087490ebf86d9e4661e1b9b
I20260808 07:14:41.991429 27583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:42.004217 27583 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:42.004679 27583 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:42.012538 27583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb: Generated new TSK 0
I20260808 07:14:42.012706 27583 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:42.018046 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.65:0
--local_ip_for_outbound_sockets=127.30.54.65
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34813
--builtin_ntp_servers=127.30.54.84:45591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:14:42.101212 27595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:42.101410 27595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:42.101431 27595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:42.102538 27595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:42.102578 27595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:42.102602 27595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.65
I20260808 07:14:42.104017 27595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:45591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.54.65
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.65
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:42.104265 27595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:42.104573 27595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:42.110376 27602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:42.110432 27603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:42.110720 27595 server_base.cc:1034] running on GCE node
W20260808 07:14:42.110762 27606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:42.111028 27595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:42.111291 27595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:42.112449 27595 hybrid_clock.cc:648] HybridClock initialized: now 1786173282112432 us; error 32 us; skew 500 ppm
I20260808 07:14:42.113860 27595 webserver.cc:460] Webserver started at http://127.30.54.65:42129/ using document root <none> and password file <none>
I20260808 07:14:42.114068 27595 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:42.114110 27595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:42.114207 27595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:42.115244 27595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/data/instance:
uuid: "5655079939e445b7b842d2d2a374809a"
format_stamp: "Formatted at 2026-08-08 07:14:42 on dist-test-slave-znh6"
server_key: "fe8e5e6bdcff1ee23c6e77f4a33c6e55"
server_key_iv: "ce2b81375fb7cafa132725c0ca6a8b98"
server_key_version: "encryptionkey@0"
I20260808 07:14:42.115547 27595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance:
uuid: "5655079939e445b7b842d2d2a374809a"
format_stamp: "Formatted at 2026-08-08 07:14:42 on dist-test-slave-znh6"
server_key: "fe8e5e6bdcff1ee23c6e77f4a33c6e55"
server_key_iv: "ce2b81375fb7cafa132725c0ca6a8b98"
server_key_version: "encryptionkey@0"
I20260808 07:14:42.116799 27595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:14:42.117518 27613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:42.117729 27595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:42.117841 27595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/wal
uuid: "5655079939e445b7b842d2d2a374809a"
format_stamp: "Formatted at 2026-08-08 07:14:42 on dist-test-slave-znh6"
server_key: "fe8e5e6bdcff1ee23c6e77f4a33c6e55"
server_key_iv: "ce2b81375fb7cafa132725c0ca6a8b98"
server_key_version: "encryptionkey@0"
I20260808 07:14:42.117929 27595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:42.131923 27595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:42.132200 27595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:42.132360 27595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:42.132571 27595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:42.132886 27595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:42.132942 27595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:42.132985 27595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:42.133014 27595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:42.139205 27595 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.65:38735
I20260808 07:14:42.139245 27761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.65:38735 every 8 connection(s)
I20260808 07:14:42.139572 27595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/data/info.pb
I20260808 07:14:42.143555 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 27595
I20260808 07:14:42.143684 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-0/wal/instance
I20260808 07:14:42.143922 30937 external_mini_cluster.cc:1428] Setting key d4a47441f6d534c816445dde8916447f
I20260808 07:14:42.145033 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.140273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.65:42777 (local address 127.30.54.126:34813)
0808 07:14:42.140317 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.140319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.140552 (+ 233us) server_negotiation.cc:407] Connection header received
0808 07:14:42.141065 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.141066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.141082 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.141148 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:42.141810 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.142342 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.143431 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.143636 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.143713 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.144721 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:42.144722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:42.144723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:42.144728 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:42.144747 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:42.144752 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:42.144811 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.144883 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.144918 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:14:42.145046 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.66:0
--local_ip_for_outbound_sockets=127.30.54.66
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34813
--builtin_ntp_servers=127.30.54.84:45591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:14:42.145126 27766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.140251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34813 (local address 127.30.54.65:42777)
0808 07:14:42.140439 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:42.140471 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:42.140964 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:42.141211 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:42.141214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:42.141224 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:42.141720 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.141723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.142442 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.142443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.143297 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.143299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.143749 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.143750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.143925 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.144532 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:14:42.144532 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:42.144582 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:14:42.144615 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:42.144811 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:42.144814 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:42.144817 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:14:42.144867 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:14:42.144945 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260808 07:14:42.145650 27764 heartbeater.cc:347] Connected to a master server at 127.30.54.126:34813
I20260808 07:14:42.145747 27764 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:42.145941 27764 heartbeater.cc:511] Master 127.30.54.126:34813 requested a full tablet report, sending...
I20260808 07:14:42.146457 27509 ts_manager.cc:194] Registered new tserver with Master: 5655079939e445b7b842d2d2a374809a (127.30.54.65:38735)
I20260808 07:14:42.147230 27509 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.65:42777
W20260808 07:14:42.225775 27771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:42.225965 27771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:42.225986 27771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:42.227128 27771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:42.227165 27771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:42.227190 27771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.66
I20260808 07:14:42.228519 27771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:45591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.54.66
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.66
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:42.228746 27771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:42.229020 27771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:42.233987 27779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:42.233989 27782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:42.234005 27777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:42.234130 27771 server_base.cc:1034] running on GCE node
I20260808 07:14:42.234372 27771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:42.234620 27771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:42.235772 27771 hybrid_clock.cc:648] HybridClock initialized: now 1786173282235736 us; error 50 us; skew 500 ppm
I20260808 07:14:42.236965 27771 webserver.cc:460] Webserver started at http://127.30.54.66:34695/ using document root <none> and password file <none>
I20260808 07:14:42.237166 27771 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:42.237208 27771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:42.237299 27771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:42.238260 27771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/data/instance:
uuid: "20806576e34e42aaa4da81f24ba24128"
format_stamp: "Formatted at 2026-08-08 07:14:42 on dist-test-slave-znh6"
server_key: "3953a207067fb763f2c5c688a1b8f8ce"
server_key_iv: "b5179e07b19a402478af22fd0f854718"
server_key_version: "encryptionkey@0"
I20260808 07:14:42.238581 27771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance:
uuid: "20806576e34e42aaa4da81f24ba24128"
format_stamp: "Formatted at 2026-08-08 07:14:42 on dist-test-slave-znh6"
server_key: "3953a207067fb763f2c5c688a1b8f8ce"
server_key_iv: "b5179e07b19a402478af22fd0f854718"
server_key_version: "encryptionkey@0"
I20260808 07:14:42.239872 27771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:14:42.240557 27793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:42.240724 27771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:42.240808 27771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/wal
uuid: "20806576e34e42aaa4da81f24ba24128"
format_stamp: "Formatted at 2026-08-08 07:14:42 on dist-test-slave-znh6"
server_key: "3953a207067fb763f2c5c688a1b8f8ce"
server_key_iv: "b5179e07b19a402478af22fd0f854718"
server_key_version: "encryptionkey@0"
I20260808 07:14:42.240861 27771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:42.275259 27771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:42.275532 27771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:42.275635 27771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:42.275821 27771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:42.276165 27771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:42.276207 27771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:42.276249 27771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:42.276278 27771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:42.282109 27771 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.66:46755
I20260808 07:14:42.282152 27957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.66:46755 every 8 connection(s)
I20260808 07:14:42.282477 27771 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/data/info.pb
I20260808 07:14:42.287403 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.283181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:59131 (local address 127.30.54.126:34813)
0808 07:14:42.283245 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.283247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.283477 (+ 230us) server_negotiation.cc:407] Connection header received
0808 07:14:42.283956 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.283957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.283968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.284029 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:42.284692 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.285190 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.286115 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.286302 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.286358 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.287180 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:42.287181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:42.287182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:42.287186 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:42.287199 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:42.287203 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:42.287241 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.287317 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.287343 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:14:42.287631 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.283169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34813 (local address 127.30.54.66:59131)
0808 07:14:42.283358 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:42.283391 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:42.283852 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:42.284103 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:42.284106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:42.284114 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:42.284608 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.284611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.285266 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.285267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.286041 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.286043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.286376 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.286377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.286482 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.287045 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:14:42.287045 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:42.287075 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:14:42.287098 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:42.287253 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:42.287257 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:42.287259 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:14:42.287312 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:14:42.287391 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260808 07:14:42.288168 27960 heartbeater.cc:347] Connected to a master server at 127.30.54.126:34813
I20260808 07:14:42.288266 27960 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:42.288481 27960 heartbeater.cc:511] Master 127.30.54.126:34813 requested a full tablet report, sending...
I20260808 07:14:42.288846 27509 ts_manager.cc:194] Registered new tserver with Master: 20806576e34e42aaa4da81f24ba24128 (127.30.54.66:46755)
I20260808 07:14:42.289263 27509 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.66:59131
I20260808 07:14:42.291222 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 27771
I20260808 07:14:42.291325 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/wal/instance
I20260808 07:14:42.291533 30937 external_mini_cluster.cc:1428] Setting key 1379882d2c559d49d8efeca28b92d2e4
I20260808 07:14:42.292363 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.54.67:0
--local_ip_for_outbound_sockets=127.30.54.67
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34813
--builtin_ntp_servers=127.30.54.84:45591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:14:42.372663 27966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:42.372864 27966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:42.372886 27966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:42.374063 27966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:42.374104 27966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:42.374147 27966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.54.67
I20260808 07:14:42.375556 27966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:45591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.54.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.54.67
--webserver_port=0
--tserver_master_addrs=127.30.54.126:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.54.67
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:42.375784 27966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:42.376078 27966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:42.381170 27975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:42.381160 27979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:42.381275 27966 server_base.cc:1034] running on GCE node
W20260808 07:14:42.381161 27973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:42.381616 27966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:42.381904 27966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:42.383060 27966 hybrid_clock.cc:648] HybridClock initialized: now 1786173282383053 us; error 36 us; skew 500 ppm
I20260808 07:14:42.384352 27966 webserver.cc:460] Webserver started at http://127.30.54.67:45645/ using document root <none> and password file <none>
I20260808 07:14:42.384605 27966 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:42.384675 27966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:42.384827 27966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:42.385851 27966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/data/instance:
uuid: "d78765f2ae534e309884ce538e986978"
format_stamp: "Formatted at 2026-08-08 07:14:42 on dist-test-slave-znh6"
server_key: "b69c8672e9faffb2a8ea813d736aee94"
server_key_iv: "a34e0efc7c941a1187823a6f54509be2"
server_key_version: "encryptionkey@0"
I20260808 07:14:42.386209 27966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance:
uuid: "d78765f2ae534e309884ce538e986978"
format_stamp: "Formatted at 2026-08-08 07:14:42 on dist-test-slave-znh6"
server_key: "b69c8672e9faffb2a8ea813d736aee94"
server_key_iv: "a34e0efc7c941a1187823a6f54509be2"
server_key_version: "encryptionkey@0"
I20260808 07:14:42.387550 27966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:14:42.388339 27986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:42.388499 27966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:14:42.388592 27966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/wal
uuid: "d78765f2ae534e309884ce538e986978"
format_stamp: "Formatted at 2026-08-08 07:14:42 on dist-test-slave-znh6"
server_key: "b69c8672e9faffb2a8ea813d736aee94"
server_key_iv: "a34e0efc7c941a1187823a6f54509be2"
server_key_version: "encryptionkey@0"
I20260808 07:14:42.388692 27966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:42.407541 27966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:42.407809 27966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:42.407964 27966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:42.408201 27966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:14:42.408511 27966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:14:42.408562 27966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:42.408603 27966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:14:42.408629 27966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:42.414434 27966 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.67:45037
I20260808 07:14:42.414480 28162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.67:45037 every 8 connection(s)
I20260808 07:14:42.414808 27966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/data/info.pb
I20260808 07:14:42.417688 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 27966
I20260808 07:14:42.417794 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/wal/instance
I20260808 07:14:42.417979 30937 external_mini_cluster.cc:1428] Setting key 9cb6ac58c3d0d59882c0ab175940c4be
I20260808 07:14:42.420085 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.415582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.67:52435 (local address 127.30.54.126:34813)
0808 07:14:42.415638 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.415640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.415857 (+ 217us) server_negotiation.cc:407] Connection header received
0808 07:14:42.416321 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.416322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.416332 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.416377 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:42.416978 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.417493 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.418764 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.419072 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.419130 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.419897 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:42.419898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:42.419898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:42.419902 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:42.419913 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:42.419917 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:42.419956 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.420009 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.420030 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:14:42.420333 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.415551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:34813 (local address 127.30.54.67:52435)
0808 07:14:42.415724 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:42.415773 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:42.416225 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:42.416428 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:42.416431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:42.416438 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:14:42.416914 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.416917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.417578 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.417579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.418699 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.418701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.419145 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.419146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.419253 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.419770 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:14:42.419771 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:14:42.419799 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:14:42.419820 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:14:42.419958 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:14:42.419961 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:14:42.419963 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:14:42.420004 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 07:14:42.420075 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260808 07:14:42.420790 28163 heartbeater.cc:347] Connected to a master server at 127.30.54.126:34813
I20260808 07:14:42.420876 28163 heartbeater.cc:464] Registering TS with master...
I20260808 07:14:42.421048 28163 heartbeater.cc:511] Master 127.30.54.126:34813 requested a full tablet report, sending...
I20260808 07:14:42.421465 27509 ts_manager.cc:194] Registered new tserver with Master: d78765f2ae534e309884ce538e986978 (127.30.54.67:45037)
I20260808 07:14:42.421836 27509 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.54.67:52435
I20260808 07:14:42.429751 30937 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:14:42.433732 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.431142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35096 (local address 127.30.54.126:34813)
0808 07:14:42.431213 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.431215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.431261 (+ 46us) server_negotiation.cc:407] Connection header received
0808 07:14:42.431327 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.431327 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.431335 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.431381 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:42.431532 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.431975 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.432738 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.432911 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.432964 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.433500 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:42.433501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:42.433502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:42.433505 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:42.433518 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:42.433521 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:42.433562 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.433605 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.433624 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:14:42.435278 27509 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:14:42.436956 27509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:14:42.443239 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.439115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.66:46755 (local address 127.0.0.1:41696)
0808 07:14:42.439263 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:42.439301 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:42.439363 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:42.439644 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:42.439647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:42.439649 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:42.439716 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.439718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.440834 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.440835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.442140 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.442143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.442986 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.442987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.443068 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.443079 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:14:42.443124 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:14:42.443163 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":61,"threads_started":1}
I20260808 07:14:42.443351 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.439144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:45037 (local address 127.0.0.1:51294)
0808 07:14:42.439574 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:42.439586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:42.439637 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:42.440181 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:42.440182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:42.440183 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:42.440221 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.440222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.441248 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.441249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.442121 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.442124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.443105 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.443106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.443170 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.443174 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:42.443227 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:14:42.443276 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":74,"threads_started":1}
I20260808 07:14:42.443615 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.439418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.30.54.67:45037)
0808 07:14:42.440036 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.440039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.440048 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:14:42.440081 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.440083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.440097 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.440183 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:42.440336 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.441166 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.442204 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.442919 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.442988 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.443069 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.443274 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.443535 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":101,"threads_started":1}
I20260808 07:14:42.443778 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.439221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.30.54.66:46755)
0808 07:14:42.439389 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.439394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.439425 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:14:42.439470 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.439473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.439489 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.439563 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:42.439817 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.440750 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.442239 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.442808 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.442930 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.443613 (+ 683us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.443641 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.443683 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260808 07:14:42.444377 28053 tablet_service.cc:1467] Processing CreateTablet for tablet 24e2e599516142fdbfbff1409f8dc46b (DEFAULT_TABLE table=test-table [id=e29efcd7b74243fea2a1cc51f329aaf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:42.444424 27864 tablet_service.cc:1467] Processing CreateTablet for tablet 24e2e599516142fdbfbff1409f8dc46b (DEFAULT_TABLE table=test-table [id=e29efcd7b74243fea2a1cc51f329aaf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:42.444733 28053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e2e599516142fdbfbff1409f8dc46b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:42.444733 27864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e2e599516142fdbfbff1409f8dc46b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:42.444814 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.439336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:38735 (local address 127.0.0.1:53162)
0808 07:14:42.439601 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:42.439613 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:42.439660 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:42.440242 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:42.440243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:42.440245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:42.440287 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.440289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.442436 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.442437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.443658 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.443661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.444578 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.444579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.444636 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.444640 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:42.444704 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:14:42.444746 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":84,"threads_started":1}
I20260808 07:14:42.444831 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.439564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.30.54.65:38735)
0808 07:14:42.439757 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.439761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.439785 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:14:42.439825 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.439827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.439841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.439920 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:42.441288 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.442358 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.443772 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.444498 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.444577 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.444662 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.444719 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.444755 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
I20260808 07:14:42.445436 27671 tablet_service.cc:1467] Processing CreateTablet for tablet 24e2e599516142fdbfbff1409f8dc46b (DEFAULT_TABLE table=test-table [id=e29efcd7b74243fea2a1cc51f329aaf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:14:42.445688 27671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e2e599516142fdbfbff1409f8dc46b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:42.448158 28191 tablet_bootstrap.cc:492] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: Bootstrap starting.
I20260808 07:14:42.448685 28194 tablet_bootstrap.cc:492] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: Bootstrap starting.
I20260808 07:14:42.448953 28191 tablet_bootstrap.cc:654] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:42.449292 28191 log.cc:824] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:42.449342 28194 tablet_bootstrap.cc:654] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:42.449668 28194 log.cc:824] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:42.449668 28193 tablet_bootstrap.cc:492] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a: Bootstrap starting.
I20260808 07:14:42.450079 28191 tablet_bootstrap.cc:492] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: No bootstrap required, opened a new log
I20260808 07:14:42.450155 28191 ts_tablet_manager.cc:1397] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:14:42.450390 28194 tablet_bootstrap.cc:492] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: No bootstrap required, opened a new log
I20260808 07:14:42.450469 28194 ts_tablet_manager.cc:1397] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:14:42.451470 28193 tablet_bootstrap.cc:654] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:42.451753 28193 log.cc:824] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:42.452392 28193 tablet_bootstrap.cc:492] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a: No bootstrap required, opened a new log
I20260808 07:14:42.452467 28193 ts_tablet_manager.cc:1397] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:14:42.452831 28191 raft_consensus.cc:348] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } }
I20260808 07:14:42.453016 28191 raft_consensus.cc:374] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:42.453053 28191 raft_consensus.cc:729] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78765f2ae534e309884ce538e986978, State: Initialized, Role: FOLLOWER
I20260808 07:14:42.452982 28194 raft_consensus.cc:348] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } }
I20260808 07:14:42.453114 28194 raft_consensus.cc:374] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:42.453148 28194 raft_consensus.cc:729] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20806576e34e42aaa4da81f24ba24128, State: Initialized, Role: FOLLOWER
I20260808 07:14:42.453197 28191 consensus_queue.cc:260] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } }
I20260808 07:14:42.453250 28194 consensus_queue.cc:260] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } }
I20260808 07:14:42.453478 28194 ts_tablet_manager.cc:1428] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:42.453529 27960 heartbeater.cc:503] Master 127.30.54.126:34813 was elected leader, sending a full tablet report...
I20260808 07:14:42.453477 28191 ts_tablet_manager.cc:1428] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260808 07:14:42.453858 28163 heartbeater.cc:503] Master 127.30.54.126:34813 was elected leader, sending a full tablet report...
I20260808 07:14:42.454936 28193 raft_consensus.cc:348] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } }
I20260808 07:14:42.455083 28193 raft_consensus.cc:374] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:42.455118 28193 raft_consensus.cc:729] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5655079939e445b7b842d2d2a374809a, State: Initialized, Role: FOLLOWER
I20260808 07:14:42.455225 28193 consensus_queue.cc:260] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } }
I20260808 07:14:42.455514 28193 ts_tablet_manager.cc:1428] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:14:42.455533 27764 heartbeater.cc:503] Master 127.30.54.126:34813 was elected leader, sending a full tablet report...
W20260808 07:14:42.533001 27961 tablet.cc:2369] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:14:42.577819 28201 raft_consensus.cc:482] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:14:42.577988 28201 raft_consensus.cc:504] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } }
I20260808 07:14:42.578373 28201 leader_election.cc:288] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d78765f2ae534e309884ce538e986978 (127.30.54.67:45037), 5655079939e445b7b842d2d2a374809a (127.30.54.65:38735)
I20260808 07:14:42.581471 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.578572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.67:45037 (local address 127.30.54.66:44923)
0808 07:14:42.578643 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:42.578655 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:42.578713 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:42.578916 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:42.578917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:42.578919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:42.579014 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.579017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.579720 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.579721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.580517 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.580519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.581180 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.581181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.581252 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.581256 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:42.581322 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:14:42.581401 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":24}
I20260808 07:14:42.581472 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.578647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:44923 (local address 127.30.54.67:45037)
0808 07:14:42.578703 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.578705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.578717 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:14:42.578814 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.578815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.578844 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.578899 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:42.579097 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.579641 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.580625 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.581106 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.581162 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.581169 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.581341 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.581379 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:14:42.581610 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.578572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:38735 (local address 127.30.54.66:36843)
0808 07:14:42.578881 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:42.578893 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:14:42.578969 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:14:42.579102 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:14:42.579103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:14:42.579105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:14:42.579171 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.579172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.579788 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.579789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.580694 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:14:42.580696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.581346 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.581347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:14:42.581420 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.581424 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:14:42.581482 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 07:14:42.581524 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":99,"thread_start_us":108,"threads_started":1}
I20260808 07:14:42.581638 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.578641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.54.66:36843 (local address 127.30.54.65:38735)
0808 07:14:42.578703 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.578706 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.578961 (+ 255us) server_negotiation.cc:407] Connection header received
0808 07:14:42.579018 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.579019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.579033 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.579084 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:14:42.579233 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.579710 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.580812 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.581264 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.581331 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.581338 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.581499 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.581548 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:14:42.581815 28083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e2e599516142fdbfbff1409f8dc46b" candidate_uuid: "20806576e34e42aaa4da81f24ba24128" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78765f2ae534e309884ce538e986978" is_pre_election: true
I20260808 07:14:42.581966 28083 raft_consensus.cc:2454] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20806576e34e42aaa4da81f24ba24128 in term 0.
I20260808 07:14:42.581995 27703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e2e599516142fdbfbff1409f8dc46b" candidate_uuid: "20806576e34e42aaa4da81f24ba24128" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655079939e445b7b842d2d2a374809a" is_pre_election: true
I20260808 07:14:42.582151 27703 raft_consensus.cc:2454] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20806576e34e42aaa4da81f24ba24128 in term 0.
I20260808 07:14:42.582268 27796 leader_election.cc:302] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20806576e34e42aaa4da81f24ba24128, d78765f2ae534e309884ce538e986978; no voters:
I20260808 07:14:42.582401 28201 raft_consensus.cc:2781] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:14:42.582474 28201 raft_consensus.cc:482] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:14:42.582513 28201 raft_consensus.cc:3037] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:42.583325 28201 raft_consensus.cc:504] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } }
I20260808 07:14:42.583468 28201 leader_election.cc:288] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [CANDIDATE]: Term 1 election: Requested vote from peers d78765f2ae534e309884ce538e986978 (127.30.54.67:45037), 5655079939e445b7b842d2d2a374809a (127.30.54.65:38735)
I20260808 07:14:42.583631 27703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e2e599516142fdbfbff1409f8dc46b" candidate_uuid: "20806576e34e42aaa4da81f24ba24128" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5655079939e445b7b842d2d2a374809a"
I20260808 07:14:42.583681 28083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e2e599516142fdbfbff1409f8dc46b" candidate_uuid: "20806576e34e42aaa4da81f24ba24128" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78765f2ae534e309884ce538e986978"
I20260808 07:14:42.583715 27703 raft_consensus.cc:3037] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:42.583761 28083 raft_consensus.cc:3037] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:42.584441 28083 raft_consensus.cc:2454] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20806576e34e42aaa4da81f24ba24128 in term 1.
I20260808 07:14:42.584450 27703 raft_consensus.cc:2454] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20806576e34e42aaa4da81f24ba24128 in term 1.
I20260808 07:14:42.584595 27796 leader_election.cc:302] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20806576e34e42aaa4da81f24ba24128, d78765f2ae534e309884ce538e986978; no voters:
I20260808 07:14:42.584687 28201 raft_consensus.cc:2781] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:42.584815 28201 raft_consensus.cc:686] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 1 LEADER]: Becoming Leader. State: Replica: 20806576e34e42aaa4da81f24ba24128, State: Running, Role: LEADER
I20260808 07:14:42.584916 28201 consensus_queue.cc:237] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } }
I20260808 07:14:42.585672 27507 catalog_manager.cc:5495] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20806576e34e42aaa4da81f24ba24128 (127.30.54.66). New cstate: current_term: 1 leader_uuid: "20806576e34e42aaa4da81f24ba24128" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20806576e34e42aaa4da81f24ba24128" member_type: VOTER last_known_addr { host: "127.30.54.66" port: 46755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 } health_report { overall_health: UNKNOWN } } }
I20260808 07:14:42.623311 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.620770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.30.54.66:46755)
0808 07:14:42.620823 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:42.620825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:42.620832 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:14:42.620868 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:42.620869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:42.620886 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:42.620950 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:14:42.621087 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.621572 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.622319 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:42.622579 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:42.622652 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:42.622906 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:14:42.623047 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:14:42.623130 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:42.623174 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 07:14:42.623198 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:14:42.625150 28083 raft_consensus.cc:1261] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [term 1 FOLLOWER]: Refusing update from remote peer 20806576e34e42aaa4da81f24ba24128: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:42.625162 27703 raft_consensus.cc:1261] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a [term 1 FOLLOWER]: Refusing update from remote peer 20806576e34e42aaa4da81f24ba24128: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:14:42.625495 28206 consensus_queue.cc:1035] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d78765f2ae534e309884ce538e986978" member_type: VOTER last_known_addr { host: "127.30.54.67" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:42.625612 28201 consensus_queue.cc:1035] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5655079939e445b7b842d2d2a374809a" member_type: VOTER last_known_addr { host: "127.30.54.65" port: 38735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:14:42.628571 28211 mvcc.cc:205] Tried to move back new op lower bound from 7316165765629702144 to 7316165765468323840. Current Snapshot: MvccSnapshot[applied={T|T < 7316165765629702144 or (T in {7316165765629702144})}]
I20260808 07:14:42.628697 28212 mvcc.cc:205] Tried to move back new op lower bound from 7316165765629702144 to 7316165765468323840. Current Snapshot: MvccSnapshot[applied={T|T < 7316165765629702144}]
I20260808 07:14:42.628810 28214 mvcc.cc:205] Tried to move back new op lower bound from 7316165765629702144 to 7316165765468323840. Current Snapshot: MvccSnapshot[applied={T|T < 7316165765629702144}]
I20260808 07:14:42.630177 27509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:42.630262 27509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:14:42.631028 27509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dcc19b41fb5a4b6292fec00d4a41a6fb: Sending DeleteTablet for 3 replicas of tablet 24e2e599516142fdbfbff1409f8dc46b
I20260808 07:14:42.631366 27864 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e2e599516142fdbfbff1409f8dc46b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:42 UTC) from {username='slave'} at 127.0.0.1:41696
I20260808 07:14:42.631408 28053 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e2e599516142fdbfbff1409f8dc46b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:42 UTC) from {username='slave'} at 127.0.0.1:51294
I20260808 07:14:42.631484 27671 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e2e599516142fdbfbff1409f8dc46b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:14:42 UTC) from {username='slave'} at 127.0.0.1:53162
I20260808 07:14:42.631729 28226 tablet_replica.cc:330] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: stopping tablet replica
I20260808 07:14:42.631809 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 27595
I20260808 07:14:42.631834 28226 raft_consensus.cc:2229] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:14:42.631772 28227 tablet_replica.cc:330] T 24e2e599516142fdbfbff1409f8dc46b P 5655079939e445b7b842d2d2a374809a: stopping tablet replica
I20260808 07:14:42.631923 28226 raft_consensus.cc:2258] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:42.632294 28226 ts_tablet_manager.cc:1905] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:42.634084 28226 ts_tablet_manager.cc:1918] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:42.634159 28226 log.cc:1196] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-2/wal/wals/24e2e599516142fdbfbff1409f8dc46b
I20260808 07:14:42.634402 28226 ts_tablet_manager.cc:1939] T 24e2e599516142fdbfbff1409f8dc46b P d78765f2ae534e309884ce538e986978: Deleting consensus metadata
I20260808 07:14:42.637244 27490 catalog_manager.cc:4841] TS d78765f2ae534e309884ce538e986978 (127.30.54.67:45037): tablet 24e2e599516142fdbfbff1409f8dc46b (table test-table [id=e29efcd7b74243fea2a1cc51f329aaf1]) successfully deleted
I20260808 07:14:42.639596 28224 tablet_replica.cc:330] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: stopping tablet replica
I20260808 07:14:42.639675 28224 raft_consensus.cc:2229] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:14:42.639742 28224 raft_consensus.cc:2258] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:14:42.641147 28224 ts_tablet_manager.cc:1905] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:14:42.643601 28224 ts_tablet_manager.cc:1918] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:14:42.643666 28224 log.cc:1196] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: Deleting WAL directory at /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786173009519897-30937-0/minicluster-data/ts-1/wal/wals/24e2e599516142fdbfbff1409f8dc46b
I20260808 07:14:42.643885 28224 ts_tablet_manager.cc:1939] T 24e2e599516142fdbfbff1409f8dc46b P 20806576e34e42aaa4da81f24ba24128: Deleting consensus metadata
I20260808 07:14:42.647435 27489 catalog_manager.cc:4841] TS 20806576e34e42aaa4da81f24ba24128 (127.30.54.66:46755): tablet 24e2e599516142fdbfbff1409f8dc46b (table test-table [id=e29efcd7b74243fea2a1cc51f329aaf1]) successfully deleted
W20260808 07:14:42.652493 27491 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.54.65:38735 (error 108)
I20260808 07:14:42.652747 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 27771
I20260808 07:14:42.653065 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:42.652939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.65:38735 (local address 127.0.0.1:53176)
0808 07:14:42.652992 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:14:42.653031 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.65:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:14:42.653198 27491 catalog_manager.cc:4584] TS 5655079939e445b7b842d2d2a374809a (127.30.54.65:38735): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24e2e599516142fdbfbff1409f8dc46b: Network error: Client connection negotiation failed: client connection to 127.30.54.65:38735: connect: Connection refused (error 111)
I20260808 07:14:42.661620 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 27966
I20260808 07:14:42.673434 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 27467
2026-08-08T07:14:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (869 ms)
[----------] 6 tests from EncryptionPolicyTest (8816 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:42 dist-test-slave-znh6 krb5kdc[28233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:42 dist-test-slave-znh6 krb5kdc[28233](info): set up 2 sockets
Aug 08 07:14:42 dist-test-slave-znh6 krb5kdc[28233](info): commencing operation
krb5kdc: starting...
W20260808 07:14:43.879634 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:43 dist-test-slave-znh6 krb5kdc[28233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:43.997085 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42851
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:44.078598 28255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:44.078814 28255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:44.078871 28255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:44.080003 28255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:44.080067 28255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:44.080090 28255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:44.080111 28255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:44.080140 28255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:44.081591 28255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42851
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:44.081856 28255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:44.082069 28255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:44.087075 28261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:44.087076 28260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:44.087103 28263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:44.087116 28255 server_base.cc:1034] running on GCE node
I20260808 07:14:44.087563 28255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:44.087845 28255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:44.088994 28255 hybrid_clock.cc:648] HybridClock initialized: now 1786173284088982 us; error 31 us; skew 500 ppm
Aug 08 07:14:44 dist-test-slave-znh6 krb5kdc[28233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173284, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:44.095830 28255 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:44.096310 28255 webserver.cc:460] Webserver started at http://127.0.0.1:39717/ using document root <none> and password file <none>
I20260808 07:14:44.096536 28255 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:44.096601 28255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:44.096752 28255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:44.097790 28255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "893e31229b934bb2b3faa8931f9805a7"
format_stamp: "Formatted at 2026-08-08 07:14:44 on dist-test-slave-znh6"
server_key: "2bc3e9c3c393e6496460d246edcd0da5"
server_key_iv: "b62443d58a4c613570c638de93618f61"
server_key_version: "encryptionkey@0"
I20260808 07:14:44.098145 28255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "893e31229b934bb2b3faa8931f9805a7"
format_stamp: "Formatted at 2026-08-08 07:14:44 on dist-test-slave-znh6"
server_key: "2bc3e9c3c393e6496460d246edcd0da5"
server_key_iv: "b62443d58a4c613570c638de93618f61"
server_key_version: "encryptionkey@0"
I20260808 07:14:44.099556 28255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:14:44.100292 28274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:44.100497 28255 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:44.100608 28255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "893e31229b934bb2b3faa8931f9805a7"
format_stamp: "Formatted at 2026-08-08 07:14:44 on dist-test-slave-znh6"
server_key: "2bc3e9c3c393e6496460d246edcd0da5"
server_key_iv: "b62443d58a4c613570c638de93618f61"
server_key_version: "encryptionkey@0"
I20260808 07:14:44.100684 28255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:44.107609 28255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:44.109324 28255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:44.109479 28255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:44.113752 28255 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42851
I20260808 07:14:44.113821 28349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42851 every 8 connection(s)
I20260808 07:14:44.114151 28255 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:44.115038 28350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:44.117218 28350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7: Bootstrap starting.
I20260808 07:14:44.117798 28350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:44.118093 28350 log.cc:824] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:44.118785 28350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7: No bootstrap required, opened a new log
I20260808 07:14:44.120410 28350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e31229b934bb2b3faa8931f9805a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42851 } }
I20260808 07:14:44.120541 28350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:44.120585 28350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 893e31229b934bb2b3faa8931f9805a7, State: Initialized, Role: FOLLOWER
I20260808 07:14:44.120724 28350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e31229b934bb2b3faa8931f9805a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42851 } }
I20260808 07:14:44.120792 28350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:44.120832 28350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:44.120879 28350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:44.121395 28350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e31229b934bb2b3faa8931f9805a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42851 } }
I20260808 07:14:44.121500 28350 leader_election.cc:302] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 893e31229b934bb2b3faa8931f9805a7; no voters:
I20260808 07:14:44.121515 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 28255
I20260808 07:14:44.121630 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:44.121671 28350 leader_election.cc:288] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:44.121692 28354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:44.121815 28354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [term 1 LEADER]: Becoming Leader. State: Replica: 893e31229b934bb2b3faa8931f9805a7, State: Running, Role: LEADER
I20260808 07:14:44.121868 30937 external_mini_cluster.cc:1428] Setting key 01e9c3e9e9b9cc634e4af86cc7e7278f
I20260808 07:14:44.121937 28350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:44.121896 28354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e31229b934bb2b3faa8931f9805a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42851 } }
I20260808 07:14:44.122370 28354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 893e31229b934bb2b3faa8931f9805a7. Latest consensus state: current_term: 1 leader_uuid: "893e31229b934bb2b3faa8931f9805a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e31229b934bb2b3faa8931f9805a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42851 } } }
I20260808 07:14:44.122467 28354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:44.122625 28359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "893e31229b934bb2b3faa8931f9805a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "893e31229b934bb2b3faa8931f9805a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42851 } } }
I20260808 07:14:44.122720 28359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:44.122956 28370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:44.123489 28370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:44.125022 28370 catalog_manager.cc:1347] Generated new cluster ID: 8b7e39b6afdc4ef68ec38095a0e824ee
I20260808 07:14:44.125077 28370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:14:44 dist-test-slave-znh6 krb5kdc[28233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:14:44.133692 28370 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:44.134371 28370 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:44.134516 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:44.122508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38631 (local address 127.0.0.1:42851)
0808 07:14:44.122706 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:44.122715 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:44.123504 (+ 789us) server_negotiation.cc:407] Connection header received
0808 07:14:44.123803 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:44.123810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:44.123867 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:44.124005 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:44.124794 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:44.126128 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:44.127259 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:44.127874 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:44.127967 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:44.130705 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:44.130710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:44.130710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:44.130724 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:44.132764 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:44.133192 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:44.133195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:44.133195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:44.133328 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:44.133575 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:44.133576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:44.133577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:44.133855 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:44.134029 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:44.134142 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 07:14:44.134248 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
I20260808 07:14:44.135747 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:44.142923 28370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 893e31229b934bb2b3faa8931f9805a7: Generated new TSK 0
I20260808 07:14:44.143095 28370 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:44.149044 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:44.137062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45977 (local address 127.0.0.1:42851)
0808 07:14:44.137093 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:44.137094 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:44.142665 (+ 5571us) server_negotiation.cc:407] Connection header received
0808 07:14:44.142762 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:44.142763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:44.142774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:44.142850 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:44.143557 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:44.144386 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:44.145097 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:44.145313 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:44.145394 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:44.146532 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:44.146533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:44.146534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:44.146541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:44.147900 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:44.148249 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:44.148251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:44.148251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:44.148307 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:44.148547 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:44.148548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:44.148548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:44.148649 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:44.148758 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:44.148887 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 07:14:44.148950 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:14:44.151033 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 28255
W20260808 07:14:44.162104 28229 connection.cc:535] client connection to 127.0.0.1:42851 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42851): Connection reset by peer (error 104)
2026-08-08T07:14:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:44 dist-test-slave-znh6 krb5kdc[28392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:44 dist-test-slave-znh6 krb5kdc[28392](info): set up 2 sockets
Aug 08 07:14:44 dist-test-slave-znh6 krb5kdc[28392](info): commencing operation
krb5kdc: starting...
W20260808 07:14:45.365975 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:45 dist-test-slave-znh6 krb5kdc[28392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:45.481455 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:45.562333 28411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:45.562541 28411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:45.562563 28411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:45.563772 28411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:45.563817 28411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:45.563831 28411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:45.563874 28411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:45.563896 28411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:45.565402 28411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43147
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:45.565615 28411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:45.565811 28411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:45.570657 28419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:45.570696 28411 server_base.cc:1034] running on GCE node
W20260808 07:14:45.570664 28422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:45.570669 28417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:45.571054 28411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:45.571362 28411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:45.572527 28411 hybrid_clock.cc:648] HybridClock initialized: now 1786173285572511 us; error 30 us; skew 500 ppm
Aug 08 07:14:45 dist-test-slave-znh6 krb5kdc[28392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173285, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:45.574972 28411 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:45.575479 28411 webserver.cc:460] Webserver started at http://127.0.0.1:43665/ using document root <none> and password file <none>
I20260808 07:14:45.575706 28411 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:45.575768 28411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:45.575901 28411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:45.576897 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "4a5330c15b6949208a3d9e59ecfe5d38"
format_stamp: "Formatted at 2026-08-08 07:14:45 on dist-test-slave-znh6"
server_key: "803b37288ce28a4893bbf3318d2788bf"
server_key_iv: "94c8f93004de7ae9d7261a220d986c21"
server_key_version: "encryptionkey@0"
I20260808 07:14:45.577251 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "4a5330c15b6949208a3d9e59ecfe5d38"
format_stamp: "Formatted at 2026-08-08 07:14:45 on dist-test-slave-znh6"
server_key: "803b37288ce28a4893bbf3318d2788bf"
server_key_iv: "94c8f93004de7ae9d7261a220d986c21"
server_key_version: "encryptionkey@0"
I20260808 07:14:45.578563 28411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:14:45.579335 28438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:45.579514 28411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:45.579607 28411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "4a5330c15b6949208a3d9e59ecfe5d38"
format_stamp: "Formatted at 2026-08-08 07:14:45 on dist-test-slave-znh6"
server_key: "803b37288ce28a4893bbf3318d2788bf"
server_key_iv: "94c8f93004de7ae9d7261a220d986c21"
server_key_version: "encryptionkey@0"
I20260808 07:14:45.579689 28411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:45.607302 28411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:45.609288 28411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:45.609465 28411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:45.613687 28411 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43147
I20260808 07:14:45.613727 28507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43147 every 8 connection(s)
I20260808 07:14:45.614101 28411 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:45.615021 28508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:45.615962 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 28411
I20260808 07:14:45.616114 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:45.616321 30937 external_mini_cluster.cc:1428] Setting key aa111d02a6c8a062b991d91ba70da295
I20260808 07:14:45.617695 28508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38: Bootstrap starting.
I20260808 07:14:45.618327 28508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:45.618631 28508 log.cc:824] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:45.619457 28508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38: No bootstrap required, opened a new log
I20260808 07:14:45.621776 28508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5330c15b6949208a3d9e59ecfe5d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } }
I20260808 07:14:45.621930 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:45.616808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59229 (local address 127.0.0.1:43147)
0808 07:14:45.616975 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:45.616982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:45.617005 (+ 23us) server_negotiation.cc:407] Connection header received
0808 07:14:45.617454 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:45.617460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:45.617506 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:45.617617 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:45.617891 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:45.618961 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:45.620078 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:45.620481 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:45.620560 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:45.621246 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:45.621250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:45.621250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:45.621403 (+ 153us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:45.621435 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:45.621445 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:45.621515 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:45.621645 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 07:14:45.621679 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260808 07:14:45.621974 28508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:45.622021 28508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5330c15b6949208a3d9e59ecfe5d38, State: Initialized, Role: FOLLOWER
I20260808 07:14:45.622169 28508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5330c15b6949208a3d9e59ecfe5d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } }
I20260808 07:14:45.622254 28508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:45.622285 28508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:45.622330 28508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:45.622934 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:45.623044 28508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5330c15b6949208a3d9e59ecfe5d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } }
I20260808 07:14:45.623144 28508 leader_election.cc:302] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a5330c15b6949208a3d9e59ecfe5d38; no voters:
I20260808 07:14:45.623301 28508 leader_election.cc:288] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:45.623347 28515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:45.623497 28515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [term 1 LEADER]: Becoming Leader. State: Replica: 4a5330c15b6949208a3d9e59ecfe5d38, State: Running, Role: LEADER
I20260808 07:14:45.623544 28508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:45.623620 28515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5330c15b6949208a3d9e59ecfe5d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } }
I20260808 07:14:45.624018 28517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a5330c15b6949208a3d9e59ecfe5d38. Latest consensus state: current_term: 1 leader_uuid: "4a5330c15b6949208a3d9e59ecfe5d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5330c15b6949208a3d9e59ecfe5d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } } }
I20260808 07:14:45.624107 28517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:45.624274 28516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a5330c15b6949208a3d9e59ecfe5d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a5330c15b6949208a3d9e59ecfe5d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43147 } } }
I20260808 07:14:45.624328 28516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:45.624313 28525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:45.624936 28525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:45.626549 28525 catalog_manager.cc:1347] Generated new cluster ID: 3546f401ff144973b0a913afd5078d9d
I20260808 07:14:45.626605 28525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:45.629002 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:45.625414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35083 (local address 127.0.0.1:43147)
0808 07:14:45.625458 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:45.625460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:45.626179 (+ 719us) server_negotiation.cc:407] Connection header received
0808 07:14:45.626243 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:45.626244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:45.626255 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:45.626319 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:45.626553 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:45.627127 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:45.627797 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:45.628039 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:45.628118 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:45.628765 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:45.628766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:45.628766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:45.628770 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:45.628783 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:45.628787 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:45.628840 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:45.628901 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 07:14:45.628912 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:14:45.639153 28525 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:45.639838 28525 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:45.645025 28525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a5330c15b6949208a3d9e59ecfe5d38: Generated new TSK 0
I20260808 07:14:45.645198 28525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:45.647405 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 28411
W20260808 07:14:45.659736 28390 connection.cc:535] client connection to 127.0.0.1:43147 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43147): Connection reset by peer (error 104)
2026-08-08T07:14:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:45 dist-test-slave-znh6 krb5kdc[28552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:45 dist-test-slave-znh6 krb5kdc[28552](info): set up 2 sockets
Aug 08 07:14:45 dist-test-slave-znh6 krb5kdc[28552](info): commencing operation
krb5kdc: starting...
W20260808 07:14:46.872785 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:46 dist-test-slave-znh6 krb5kdc[28552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:46.993105 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41089
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:47.074654 28571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:47.074903 28571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:47.074954 28571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:47.076062 28571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:47.076126 28571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:47.076150 28571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:47.076181 28571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:47.076211 28571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:47.077626 28571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41089
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:47.077881 28571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:47.078099 28571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:47.082968 28579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:47.082983 28580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:47.082979 28571 server_base.cc:1034] running on GCE node
W20260808 07:14:47.083055 28582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:47.083475 28571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:47.083773 28571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:47.084914 28571 hybrid_clock.cc:648] HybridClock initialized: now 1786173287084902 us; error 29 us; skew 500 ppm
Aug 08 07:14:47 dist-test-slave-znh6 krb5kdc[28552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173287, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:47.087396 28571 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:47.087845 28571 webserver.cc:460] Webserver started at http://127.0.0.1:34995/ using document root <none> and password file <none>
I20260808 07:14:47.088084 28571 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:47.088130 28571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:47.088274 28571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:47.089272 28571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "b73eebb4be714aad9050a9c6f27e9541"
format_stamp: "Formatted at 2026-08-08 07:14:47 on dist-test-slave-znh6"
server_key: "58ab9abdc3375844d9159230d5ceb9ed"
server_key_iv: "698b935d9d892c0202261b9c369ca9c7"
server_key_version: "encryptionkey@0"
I20260808 07:14:47.089620 28571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "b73eebb4be714aad9050a9c6f27e9541"
format_stamp: "Formatted at 2026-08-08 07:14:47 on dist-test-slave-znh6"
server_key: "58ab9abdc3375844d9159230d5ceb9ed"
server_key_iv: "698b935d9d892c0202261b9c369ca9c7"
server_key_version: "encryptionkey@0"
I20260808 07:14:47.091012 28571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:14:47.091722 28597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:47.091877 28571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:47.091985 28571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "b73eebb4be714aad9050a9c6f27e9541"
format_stamp: "Formatted at 2026-08-08 07:14:47 on dist-test-slave-znh6"
server_key: "58ab9abdc3375844d9159230d5ceb9ed"
server_key_iv: "698b935d9d892c0202261b9c369ca9c7"
server_key_version: "encryptionkey@0"
I20260808 07:14:47.092058 28571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:47.095273 28571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:47.097028 28571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:47.097177 28571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:47.101110 28571 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41089
I20260808 07:14:47.101140 28675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41089 every 8 connection(s)
I20260808 07:14:47.101444 28571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:47.102387 28677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:47.104557 28677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541: Bootstrap starting.
I20260808 07:14:47.105105 28677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:47.105391 28677 log.cc:824] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:47.106065 28677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541: No bootstrap required, opened a new log
I20260808 07:14:47.108001 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 28571
I20260808 07:14:47.108115 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:47.108108 28677 raft_consensus.cc:348] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73eebb4be714aad9050a9c6f27e9541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260808 07:14:47.108276 28677 raft_consensus.cc:374] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:47.108300 30937 external_mini_cluster.cc:1428] Setting key 7281b097e91d726ef33fb81affe493c7
I20260808 07:14:47.108325 28677 raft_consensus.cc:729] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b73eebb4be714aad9050a9c6f27e9541, State: Initialized, Role: FOLLOWER
I20260808 07:14:47.108459 28677 consensus_queue.cc:260] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73eebb4be714aad9050a9c6f27e9541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260808 07:14:47.108533 28677 raft_consensus.cc:388] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:47.108574 28677 raft_consensus.cc:482] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:47.108620 28677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:47.109349 28677 raft_consensus.cc:504] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73eebb4be714aad9050a9c6f27e9541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260808 07:14:47.109486 28677 leader_election.cc:302] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b73eebb4be714aad9050a9c6f27e9541; no voters:
I20260808 07:14:47.109648 28677 leader_election.cc:288] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:47.109678 28684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:47.109818 28684 raft_consensus.cc:686] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [term 1 LEADER]: Becoming Leader. State: Replica: b73eebb4be714aad9050a9c6f27e9541, State: Running, Role: LEADER
I20260808 07:14:47.109952 28684 consensus_queue.cc:237] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73eebb4be714aad9050a9c6f27e9541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } }
I20260808 07:14:47.109957 28677 sys_catalog.cc:564] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:47.110476 28687 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b73eebb4be714aad9050a9c6f27e9541. Latest consensus state: current_term: 1 leader_uuid: "b73eebb4be714aad9050a9c6f27e9541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73eebb4be714aad9050a9c6f27e9541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } } }
I20260808 07:14:47.110565 28687 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:47.110693 28685 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b73eebb4be714aad9050a9c6f27e9541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73eebb4be714aad9050a9c6f27e9541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41089 } } }
I20260808 07:14:47.110752 28685 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:47.110948 28689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:47.111205 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:47.108921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36093 (local address 127.0.0.1:41089)
0808 07:14:47.109483 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:47.109492 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:47.109524 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:14:47.109645 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:47.109651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:47.109679 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:47.110056 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:47.110526 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:47.110529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:47.110530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:47.110534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:47.110597 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:47.110614 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:47.110925 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:47.110957 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 07:14:47.111000 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":441,"threads_started":1}
I20260808 07:14:47.111670 28689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:47.112128 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:47.113442 28689 catalog_manager.cc:1347] Generated new cluster ID: 3bd96a825b6a4c1bae55761107039b9f
I20260808 07:14:47.113493 28689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:47.114583 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:47.113428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41923 (local address 127.0.0.1:41089)
0808 07:14:47.113465 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:47.113467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:47.113665 (+ 198us) server_negotiation.cc:407] Connection header received
0808 07:14:47.113736 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:47.113737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:47.113758 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:47.113814 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:47.114374 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:47.114376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:47.114376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:47.114380 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:47.114392 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:47.114396 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:47.114460 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:47.114506 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:14:47.114520 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:14:47.125365 28689 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:47.125519 28689 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:47.130505 28689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b73eebb4be714aad9050a9c6f27e9541: Generated new TSK 0
I20260808 07:14:47.130667 28689 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:47.135320 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 28571
2026-08-08T07:14:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:47 dist-test-slave-znh6 krb5kdc[28718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:47 dist-test-slave-znh6 krb5kdc[28718](info): set up 2 sockets
Aug 08 07:14:47 dist-test-slave-znh6 krb5kdc[28718](info): commencing operation
krb5kdc: starting...
W20260808 07:14:48.360590 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:48 dist-test-slave-znh6 krb5kdc[28718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:48.484097 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:48.563849 28741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:48.564081 28741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:48.564119 28741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:48.565228 28741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:48.565300 28741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:48.565326 28741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:48.565356 28741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:48.565387 28741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:48.566856 28741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:48.567111 28741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:48.567337 28741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:48.572182 28749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:48.572245 28741 server_base.cc:1034] running on GCE node
W20260808 07:14:48.572199 28753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:48.572209 28747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:48.572618 28741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:48.572918 28741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:48.574100 28741 hybrid_clock.cc:648] HybridClock initialized: now 1786173288574084 us; error 41 us; skew 500 ppm
Aug 08 07:14:48 dist-test-slave-znh6 krb5kdc[28718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173288, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:48.576761 28741 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:48.577135 28741 webserver.cc:460] Webserver started at http://127.0.0.1:35047/ using document root <none> and password file <none>
I20260808 07:14:48.577337 28741 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:48.577378 28741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:48.577463 28741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:48.578480 28741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "c4938155753b40f694d7fd7c04043f47"
format_stamp: "Formatted at 2026-08-08 07:14:48 on dist-test-slave-znh6"
server_key: "97dbc5c6cd970c78ea3e563875573938"
server_key_iv: "495134fc438fa32e589b2de7c1112d84"
server_key_version: "encryptionkey@0"
I20260808 07:14:48.578771 28741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "c4938155753b40f694d7fd7c04043f47"
format_stamp: "Formatted at 2026-08-08 07:14:48 on dist-test-slave-znh6"
server_key: "97dbc5c6cd970c78ea3e563875573938"
server_key_iv: "495134fc438fa32e589b2de7c1112d84"
server_key_version: "encryptionkey@0"
I20260808 07:14:48.580090 28741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:48.580834 28764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:48.581040 28741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:48.581146 28741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "c4938155753b40f694d7fd7c04043f47"
format_stamp: "Formatted at 2026-08-08 07:14:48 on dist-test-slave-znh6"
server_key: "97dbc5c6cd970c78ea3e563875573938"
server_key_iv: "495134fc438fa32e589b2de7c1112d84"
server_key_version: "encryptionkey@0"
I20260808 07:14:48.581233 28741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:48.590169 28741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:48.592375 28741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:48.592526 28741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:48.596567 28741 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41675
I20260808 07:14:48.596632 28846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41675 every 8 connection(s)
I20260808 07:14:48.596935 28741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:48.597775 28847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:48.598666 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 28741
I20260808 07:14:48.598801 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:48.599025 30937 external_mini_cluster.cc:1428] Setting key bdf1efece7bd2652c0147c125f7d1312
I20260808 07:14:48.600502 28847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47: Bootstrap starting.
I20260808 07:14:48.601253 28847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:48.601576 28847 log.cc:824] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:48.602260 28847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47: No bootstrap required, opened a new log
I20260808 07:14:48.604630 28847 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4938155753b40f694d7fd7c04043f47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260808 07:14:48.604817 28847 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:48.604861 28847 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4938155753b40f694d7fd7c04043f47, State: Initialized, Role: FOLLOWER
I20260808 07:14:48.604993 28847 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4938155753b40f694d7fd7c04043f47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260808 07:14:48.605088 28847 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:48.605126 28847 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:48.605177 28847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:48.605885 28847 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4938155753b40f694d7fd7c04043f47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260808 07:14:48.605973 28847 leader_election.cc:302] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4938155753b40f694d7fd7c04043f47; no voters:
I20260808 07:14:48.606161 28847 leader_election.cc:288] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:48.606191 28852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [term 1 FOLLOWER]: Leader election won for term 1
Aug 08 07:14:48 dist-test-slave-znh6 krb5kdc[28718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:14:48.606333 28852 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [term 1 LEADER]: Becoming Leader. State: Replica: c4938155753b40f694d7fd7c04043f47, State: Running, Role: LEADER
I20260808 07:14:48.606417 28847 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:48.606410 28852 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4938155753b40f694d7fd7c04043f47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260808 07:14:48.606876 28854 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4938155753b40f694d7fd7c04043f47. Latest consensus state: current_term: 1 leader_uuid: "c4938155753b40f694d7fd7c04043f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4938155753b40f694d7fd7c04043f47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } } }
I20260808 07:14:48.606954 28854 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:48.607154 28860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:48.607585 28860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:48.608466 28855 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4938155753b40f694d7fd7c04043f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4938155753b40f694d7fd7c04043f47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } } }
I20260808 07:14:48.608630 28855 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:48.610880 28860 catalog_manager.cc:1347] Generated new cluster ID: 5ac8e25f35234b66ab9dfe1d64c52e51
I20260808 07:14:48.610993 28860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:48.611044 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:48.599564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57815 (local address 127.0.0.1:41675)
0808 07:14:48.599761 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:48.599772 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:48.599811 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:14:48.600408 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:48.600416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:48.600476 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:48.600657 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:48.601522 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:48.602528 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:48.603601 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:48.603989 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:48.604078 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:48.606945 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:48.606949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:48.606950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:48.606961 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:48.609278 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:48.609654 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:48.609663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:48.609663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:48.609763 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:48.609995 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:48.609996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:48.609997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:48.610309 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:48.610456 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:48.610602 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 07:14:48.610749 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260808 07:14:48.611908 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:48.621845 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:48.615097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37819 (local address 127.0.0.1:41675)
0808 07:14:48.615128 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:48.615129 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:48.615215 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:14:48.615275 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:48.615276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:48.615285 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:48.615342 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:48.615934 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:48.616623 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:48.617589 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:48.617904 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:48.617963 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:48.619276 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:48.619277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:48.619277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:48.619283 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:48.620758 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:48.621137 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:48.621137 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:48.621138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:48.621188 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:48.621380 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:48.621381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:48.621381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:48.621489 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:48.621567 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:48.621692 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 07:14:48.621763 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:14:48.632354 28860 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:48.632853 28860 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:48.635110 28860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4938155753b40f694d7fd7c04043f47: Generated new TSK 0
I20260808 07:14:48.635293 28860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:48.640049 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 28741
2026-08-08T07:14:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:48 dist-test-slave-znh6 krb5kdc[28899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:48 dist-test-slave-znh6 krb5kdc[28899](info): set up 2 sockets
Aug 08 07:14:48 dist-test-slave-znh6 krb5kdc[28899](info): commencing operation
krb5kdc: starting...
W20260808 07:14:49.845214 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:49 dist-test-slave-znh6 krb5kdc[28899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:49.961694 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:50.042245 28924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:50.042476 28924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:50.042516 28924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:50.043680 28924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:50.043747 28924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:50.043771 28924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:50.043792 28924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:50.043819 28924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:50.045261 28924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:50.045528 28924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:50.045749 28924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:50.050642 28933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:50.050652 28932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:50.050683 28924 server_base.cc:1034] running on GCE node
W20260808 07:14:50.050653 28935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:50.051067 28924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:50.051355 28924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:50.052531 28924 hybrid_clock.cc:648] HybridClock initialized: now 1786173290052513 us; error 37 us; skew 500 ppm
Aug 08 07:14:50 dist-test-slave-znh6 krb5kdc[28899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173290, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:50.062283 28924 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:50.062808 28924 webserver.cc:460] Webserver started at http://127.0.0.1:45743/ using document root <none> and password file <none>
I20260808 07:14:50.063097 28924 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:50.063164 28924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:50.063333 28924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:50.064523 28924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "8f1065fb67f34ae3ab4eccd99a836c8e"
format_stamp: "Formatted at 2026-08-08 07:14:50 on dist-test-slave-znh6"
server_key: "b119a5c25a6ade9a8536fc1bd96f5cd7"
server_key_iv: "26706ffc2743cf56ba393ea423589216"
server_key_version: "encryptionkey@0"
I20260808 07:14:50.064913 28924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "8f1065fb67f34ae3ab4eccd99a836c8e"
format_stamp: "Formatted at 2026-08-08 07:14:50 on dist-test-slave-znh6"
server_key: "b119a5c25a6ade9a8536fc1bd96f5cd7"
server_key_iv: "26706ffc2743cf56ba393ea423589216"
server_key_version: "encryptionkey@0"
I20260808 07:14:50.066298 28924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:14:50.067035 28945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:50.067219 28924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:50.067310 28924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "8f1065fb67f34ae3ab4eccd99a836c8e"
format_stamp: "Formatted at 2026-08-08 07:14:50 on dist-test-slave-znh6"
server_key: "b119a5c25a6ade9a8536fc1bd96f5cd7"
server_key_iv: "26706ffc2743cf56ba393ea423589216"
server_key_version: "encryptionkey@0"
I20260808 07:14:50.067382 28924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:50.074138 28924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:50.075938 28924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:50.076093 28924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:50.080715 28924 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43691
I20260808 07:14:50.080817 29013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43691 every 8 connection(s)
I20260808 07:14:50.081175 28924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:50.082260 29014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:50.084591 29014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e: Bootstrap starting.
I20260808 07:14:50.085119 29014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:50.085371 29014 log.cc:824] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:50.086210 29014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e: No bootstrap required, opened a new log
I20260808 07:14:50.086302 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 28924
I20260808 07:14:50.086424 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:50.086635 30937 external_mini_cluster.cc:1428] Setting key 9b338fe87040f4b0af1cd631f34576fd
I20260808 07:14:50.088191 29014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1065fb67f34ae3ab4eccd99a836c8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43691 } }
I20260808 07:14:50.088323 29014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:50.088346 29014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f1065fb67f34ae3ab4eccd99a836c8e, State: Initialized, Role: FOLLOWER
I20260808 07:14:50.088490 29014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1065fb67f34ae3ab4eccd99a836c8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43691 } }
I20260808 07:14:50.088562 29014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:50.088584 29014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:50.088650 29014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:50.089188 29014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1065fb67f34ae3ab4eccd99a836c8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43691 } }
I20260808 07:14:50.089319 29014 leader_election.cc:302] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f1065fb67f34ae3ab4eccd99a836c8e; no voters:
I20260808 07:14:50.089534 29014 leader_election.cc:288] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:50.089578 29020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:50.089756 29020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [term 1 LEADER]: Becoming Leader. State: Replica: 8f1065fb67f34ae3ab4eccd99a836c8e, State: Running, Role: LEADER
I20260808 07:14:50.089794 29014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:50.089927 29020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1065fb67f34ae3ab4eccd99a836c8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43691 } }
I20260808 07:14:50.090265 29020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f1065fb67f34ae3ab4eccd99a836c8e. Latest consensus state: current_term: 1 leader_uuid: "8f1065fb67f34ae3ab4eccd99a836c8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1065fb67f34ae3ab4eccd99a836c8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43691 } } }
I20260808 07:14:50.090348 29020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:50.090276 29021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f1065fb67f34ae3ab4eccd99a836c8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f1065fb67f34ae3ab4eccd99a836c8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43691 } } }
I20260808 07:14:50.090540 29021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:50.091449 29031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:50.091974 29031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:50.093221 29019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.087252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52211 (local address 127.0.0.1:43691)
0808 07:14:50.087435 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:50.087444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:50.087475 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:14:50.087847 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:50.087852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:50.087902 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:50.088045 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:50.088269 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:50.089727 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:50.090921 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:50.091523 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:50.091656 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:50.092390 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:50.092394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:50.092394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:50.092601 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:50.092638 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:50.092655 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:50.092718 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:50.092939 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 07:14:50.093004 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260808 07:14:50.093731 29031 catalog_manager.cc:1347] Generated new cluster ID: 0ca08af20c8d49779b7e0c6ae54399f3
I20260808 07:14:50.093797 29031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:50.094269 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:50.101629 29019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:50.095628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47649 (local address 127.0.0.1:43691)
0808 07:14:50.095664 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:50.095666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:50.099011 (+ 3345us) server_negotiation.cc:407] Connection header received
0808 07:14:50.099076 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:50.099077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:50.099087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:50.099142 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:50.099272 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:50.099738 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:50.100442 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:50.100658 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:50.100734 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:50.101384 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:50.101385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:50.101386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:50.101390 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:50.101403 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:50.101407 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:50.101453 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:50.101517 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:14:50.101538 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:14:50.105207 29031 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:50.105862 29031 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:50.108424 29031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f1065fb67f34ae3ab4eccd99a836c8e: Generated new TSK 0
I20260808 07:14:50.108579 29031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:50.115916 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 28924
2026-08-08T07:14:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1474 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:50 dist-test-slave-znh6 krb5kdc[29059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:50 dist-test-slave-znh6 krb5kdc[29059](info): set up 2 sockets
Aug 08 07:14:50 dist-test-slave-znh6 krb5kdc[29059](info): commencing operation
krb5kdc: starting...
W20260808 07:14:51.346444 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:51 dist-test-slave-znh6 krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:51.465942 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45371
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:51.546883 29077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:51.547083 29077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:51.547106 29077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:51.548159 29077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:51.548197 29077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:51.548210 29077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:51.548223 29077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:51.548234 29077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:51.549618 29077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:51.549865 29077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:51.550047 29077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:14:51.554952 29077 server_base.cc:1034] running on GCE node
W20260808 07:14:51.554956 29086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:51.554913 29090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:51.554922 29085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:51.555559 29077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:51.555840 29077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:51.557005 29077 hybrid_clock.cc:648] HybridClock initialized: now 1786173291557005 us; error 45 us; skew 500 ppm
Aug 08 07:14:51 dist-test-slave-znh6 krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173291, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:51.559641 29077 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:51.560114 29077 webserver.cc:460] Webserver started at http://127.0.0.1:35555/ using document root <none> and password file <none>
I20260808 07:14:51.560320 29077 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:51.560362 29077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:51.560457 29077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:51.561453 29077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "4b808fc94d134a13a27482eee47e52ee"
format_stamp: "Formatted at 2026-08-08 07:14:51 on dist-test-slave-znh6"
server_key: "d8b52821fbcf6ec8be0fdbb9c8eb1716"
server_key_iv: "db3eba9cb64bb0c277347499ce513edd"
server_key_version: "encryptionkey@0"
I20260808 07:14:51.561757 29077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "4b808fc94d134a13a27482eee47e52ee"
format_stamp: "Formatted at 2026-08-08 07:14:51 on dist-test-slave-znh6"
server_key: "d8b52821fbcf6ec8be0fdbb9c8eb1716"
server_key_iv: "db3eba9cb64bb0c277347499ce513edd"
server_key_version: "encryptionkey@0"
I20260808 07:14:51.563053 29077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:14:51.563767 29099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:51.563951 29077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:51.564040 29077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "4b808fc94d134a13a27482eee47e52ee"
format_stamp: "Formatted at 2026-08-08 07:14:51 on dist-test-slave-znh6"
server_key: "d8b52821fbcf6ec8be0fdbb9c8eb1716"
server_key_iv: "db3eba9cb64bb0c277347499ce513edd"
server_key_version: "encryptionkey@0"
I20260808 07:14:51.564116 29077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:51.567401 29077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:51.569255 29077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:51.569416 29077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:51.573391 29077 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45371
I20260808 07:14:51.573424 29171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45371 every 8 connection(s)
I20260808 07:14:51.573762 29077 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:51.574769 29172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:51.577042 29172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee: Bootstrap starting.
I20260808 07:14:51.577625 29172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:51.577919 29172 log.cc:824] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:51.578683 29172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee: No bootstrap required, opened a new log
I20260808 07:14:51.580312 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 29077
I20260808 07:14:51.580457 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:51.580310 29172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b808fc94d134a13a27482eee47e52ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45371 } }
I20260808 07:14:51.580587 29172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:51.580650 29172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b808fc94d134a13a27482eee47e52ee, State: Initialized, Role: FOLLOWER
I20260808 07:14:51.580711 30937 external_mini_cluster.cc:1428] Setting key f29f020bd1e544e29425f193e2c13d3c
I20260808 07:14:51.580821 29172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b808fc94d134a13a27482eee47e52ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45371 } }
I20260808 07:14:51.580943 29172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:51.580981 29172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:51.581037 29172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:51.582218 29172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b808fc94d134a13a27482eee47e52ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45371 } }
I20260808 07:14:51.582335 29172 leader_election.cc:302] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b808fc94d134a13a27482eee47e52ee; no voters:
I20260808 07:14:51.582496 29172 leader_election.cc:288] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:51.582525 29177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:51.582630 29177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [term 1 LEADER]: Becoming Leader. State: Replica: 4b808fc94d134a13a27482eee47e52ee, State: Running, Role: LEADER
I20260808 07:14:51.582756 29172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:51.582762 29177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b808fc94d134a13a27482eee47e52ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45371 } }
I20260808 07:14:51.583068 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:51.581236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52863 (local address 127.0.0.1:45371)
0808 07:14:51.581468 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:51.581478 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:51.581517 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:14:51.581651 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:51.581660 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:51.581710 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:51.581868 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:51.582511 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:51.582515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:51.582516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:51.582520 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:51.582574 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:51.582591 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:51.582659 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:51.582848 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 07:14:51.582891 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":111,"threads_started":1}
I20260808 07:14:51.583220 29178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b808fc94d134a13a27482eee47e52ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b808fc94d134a13a27482eee47e52ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45371 } } }
I20260808 07:14:51.583353 29178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:51.583297 29179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b808fc94d134a13a27482eee47e52ee. Latest consensus state: current_term: 1 leader_uuid: "4b808fc94d134a13a27482eee47e52ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b808fc94d134a13a27482eee47e52ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45371 } } }
I20260808 07:14:51.583424 29179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:51.583585 29181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:51.584034 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:51.584167 29181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:51.585898 29181 catalog_manager.cc:1347] Generated new cluster ID: a49380699e9643fb99e5d10a154da1fc
I20260808 07:14:51.585940 29181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:51.593797 29176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:51.587110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59951 (local address 127.0.0.1:45371)
0808 07:14:51.587150 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:51.587152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:51.592752 (+ 5600us) server_negotiation.cc:407] Connection header received
0808 07:14:51.592858 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:51.592859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:51.592864 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:51.592922 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:51.593565 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:51.593567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:51.593567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:51.593571 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:51.593588 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:51.593592 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:51.593655 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:51.593709 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 07:14:51.593720 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:14:51.596227 29181 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:51.596400 29181 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:51.601986 29181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b808fc94d134a13a27482eee47e52ee: Generated new TSK 0
I20260808 07:14:51.602137 29181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:51.605861 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 29077
2026-08-08T07:14:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:51 dist-test-slave-znh6 krb5kdc[29215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:51 dist-test-slave-znh6 krb5kdc[29215](info): set up 2 sockets
Aug 08 07:14:51 dist-test-slave-znh6 krb5kdc[29215](info): commencing operation
krb5kdc: starting...
W20260808 07:14:52.833633 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:52 dist-test-slave-znh6 krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:52.951540 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:53.032171 29230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:53.032393 29230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:53.032435 29230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:53.033596 29230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:53.033658 29230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:53.033684 29230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:53.033712 29230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:53.033743 29230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:53.035204 29230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:53.035476 29230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:53.035701 29230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:53.040462 29239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:53.040452 29241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:53.040548 29230 server_base.cc:1034] running on GCE node
W20260808 07:14:53.040453 29237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:53.040889 29230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:53.041168 29230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:53.042344 29230 hybrid_clock.cc:648] HybridClock initialized: now 1786173293042344 us; error 40 us; skew 500 ppm
Aug 08 07:14:53 dist-test-slave-znh6 krb5kdc[29215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173293, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:53.044744 29230 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:53.045220 29230 webserver.cc:460] Webserver started at http://127.0.0.1:41225/ using document root <none> and password file <none>
I20260808 07:14:53.045454 29230 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:53.045524 29230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:53.045674 29230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:53.046674 29230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "8c2776c261aa4f63b01252925363e65c"
format_stamp: "Formatted at 2026-08-08 07:14:53 on dist-test-slave-znh6"
server_key: "e1d70a7bfa974b223b913b4f55f44f85"
server_key_iv: "ca6e1e90b7ab5ff23678e66154dc2e9b"
server_key_version: "encryptionkey@0"
I20260808 07:14:53.047062 29230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "8c2776c261aa4f63b01252925363e65c"
format_stamp: "Formatted at 2026-08-08 07:14:53 on dist-test-slave-znh6"
server_key: "e1d70a7bfa974b223b913b4f55f44f85"
server_key_iv: "ca6e1e90b7ab5ff23678e66154dc2e9b"
server_key_version: "encryptionkey@0"
I20260808 07:14:53.048404 29230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:53.049166 29259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:53.049377 29230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:53.049492 29230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "8c2776c261aa4f63b01252925363e65c"
format_stamp: "Formatted at 2026-08-08 07:14:53 on dist-test-slave-znh6"
server_key: "e1d70a7bfa974b223b913b4f55f44f85"
server_key_iv: "ca6e1e90b7ab5ff23678e66154dc2e9b"
server_key_version: "encryptionkey@0"
I20260808 07:14:53.049566 29230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:53.072188 29230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:53.073920 29230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:53.074081 29230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:53.078212 29230 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46059
I20260808 07:14:53.078245 29330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46059 every 8 connection(s)
I20260808 07:14:53.078583 29230 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:53.079463 29331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:53.081540 29331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c: Bootstrap starting.
I20260808 07:14:53.082072 29331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:53.082302 29331 log.cc:824] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:53.082993 29331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c: No bootstrap required, opened a new log
I20260808 07:14:53.084512 29331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2776c261aa4f63b01252925363e65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260808 07:14:53.084640 29331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:53.084663 29331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c2776c261aa4f63b01252925363e65c, State: Initialized, Role: FOLLOWER
I20260808 07:14:53.084789 29331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2776c261aa4f63b01252925363e65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260808 07:14:53.084867 29331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:53.084890 29331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:53.084931 29331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:53.085386 29331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2776c261aa4f63b01252925363e65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260808 07:14:53.085467 29331 leader_election.cc:302] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c2776c261aa4f63b01252925363e65c; no voters:
I20260808 07:14:53.085582 29331 leader_election.cc:288] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:53.085629 29335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:53.085791 29335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [term 1 LEADER]: Becoming Leader. State: Replica: 8c2776c261aa4f63b01252925363e65c, State: Running, Role: LEADER
I20260808 07:14:53.085870 29331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:53.085934 29335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2776c261aa4f63b01252925363e65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260808 07:14:53.085960 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 29230
I20260808 07:14:53.086107 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:53.086321 30937 external_mini_cluster.cc:1428] Setting key cbfd2051d0bd610811bb11657fde65af
I20260808 07:14:53.086418 29337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c2776c261aa4f63b01252925363e65c. Latest consensus state: current_term: 1 leader_uuid: "8c2776c261aa4f63b01252925363e65c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2776c261aa4f63b01252925363e65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } } }
I20260808 07:14:53.086519 29337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:53.086730 29346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:53.087030 29336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c2776c261aa4f63b01252925363e65c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2776c261aa4f63b01252925363e65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } } }
I20260808 07:14:53.087193 29336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:53.087265 29346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:53.088819 29346 catalog_manager.cc:1347] Generated new cluster ID: 2a4a712423804e7586f86b1f930425b0
I20260808 07:14:53.088886 29346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:14:53 dist-test-slave-znh6 krb5kdc[29215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:14:53.096024 29346 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:53.096674 29346 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:53.098037 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:53.086860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:40417 (local address 127.0.0.1:46059)
0808 07:14:53.087740 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:53.087749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:53.087779 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:14:53.087889 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:53.087896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:53.087955 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:53.088856 (+ 901us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:53.088962 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:53.089864 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:53.090659 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:53.091272 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:53.094224 (+ 2952us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:53.094391 (+ 167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:53.094405 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:53.094405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:53.094416 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:53.096415 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:53.096852 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:53.096854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:53.096855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:53.096949 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:53.097202 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:53.097203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:53.097203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:53.097462 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:53.097601 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:53.097745 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 07:14:53.097837 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":80,"threads_started":1}
I20260808 07:14:53.098949 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:53.101060 29346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c2776c261aa4f63b01252925363e65c: Generated new TSK 0
I20260808 07:14:53.101208 29346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:53.109829 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:53.103114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:47053 (local address 127.0.0.1:46059)
0808 07:14:53.103150 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:53.103152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:53.103312 (+ 160us) server_negotiation.cc:407] Connection header received
0808 07:14:53.103365 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:53.103366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:53.103373 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:53.103428 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:53.104018 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:53.104502 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:53.105562 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:53.105829 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:53.105895 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:53.107242 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:53.107243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:53.107243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:53.107249 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:53.108652 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:53.109079 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:53.109080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:53.109080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:53.109133 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:53.109357 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:53.109357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:53.109358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:53.109458 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:53.109529 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:53.109694 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 07:14:53.109745 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:14:53.111634 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 29230
W20260808 07:14:53.122478 29212 connection.cc:535] client connection to 127.0.0.1:46059 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46059): Connection reset by peer (error 104)
2026-08-08T07:14:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:53 dist-test-slave-znh6 krb5kdc[29369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:53 dist-test-slave-znh6 krb5kdc[29369](info): set up 2 sockets
Aug 08 07:14:53 dist-test-slave-znh6 krb5kdc[29369](info): commencing operation
krb5kdc: starting...
W20260808 07:14:54.348570 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:54 dist-test-slave-znh6 krb5kdc[29369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:54.473822 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:54.553963 29389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:54.554193 29389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:54.554232 29389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:54.555382 29389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:54.555449 29389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:54.555475 29389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:54.555507 29389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:54.555536 29389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:54.556991 29389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:54.557253 29389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:54.557479 29389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:54.562320 29397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:54.562335 29399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:54.562352 29396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:54.562372 29389 server_base.cc:1034] running on GCE node
I20260808 07:14:54.562860 29389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:54.563153 29389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:54.564327 29389 hybrid_clock.cc:648] HybridClock initialized: now 1786173294564319 us; error 45 us; skew 500 ppm
Aug 08 07:14:54 dist-test-slave-znh6 krb5kdc[29369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173294, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:54.566859 29389 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:54.567287 29389 webserver.cc:460] Webserver started at http://127.0.0.1:42033/ using document root <none> and password file <none>
I20260808 07:14:54.567519 29389 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:54.567587 29389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:54.567739 29389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:54.568790 29389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "8e05600cc38440c2aecc865a0730e0cd"
format_stamp: "Formatted at 2026-08-08 07:14:54 on dist-test-slave-znh6"
server_key: "7bc486bce4db4d926e8627b6ddf3ffe1"
server_key_iv: "fe412dd4f5c929cde1b1a22e9df30222"
server_key_version: "encryptionkey@0"
I20260808 07:14:54.569164 29389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "8e05600cc38440c2aecc865a0730e0cd"
format_stamp: "Formatted at 2026-08-08 07:14:54 on dist-test-slave-znh6"
server_key: "7bc486bce4db4d926e8627b6ddf3ffe1"
server_key_iv: "fe412dd4f5c929cde1b1a22e9df30222"
server_key_version: "encryptionkey@0"
I20260808 07:14:54.570457 29389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:14:54.571259 29411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:54.571576 29389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:14:54.571677 29389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "8e05600cc38440c2aecc865a0730e0cd"
format_stamp: "Formatted at 2026-08-08 07:14:54 on dist-test-slave-znh6"
server_key: "7bc486bce4db4d926e8627b6ddf3ffe1"
server_key_iv: "fe412dd4f5c929cde1b1a22e9df30222"
server_key_version: "encryptionkey@0"
I20260808 07:14:54.571759 29389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:54.600596 29389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:54.602452 29389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:54.602622 29389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:54.606513 29389 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37861
I20260808 07:14:54.606555 29482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37861 every 8 connection(s)
I20260808 07:14:54.606992 29389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:54.607921 29484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:54.608428 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 29389
I20260808 07:14:54.608563 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:54.608784 30937 external_mini_cluster.cc:1428] Setting key 51eeac96cef167b844ac0d9cf7d9d5cb
I20260808 07:14:54.610895 29484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd: Bootstrap starting.
I20260808 07:14:54.611586 29484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:54.611900 29484 log.cc:824] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:54.612619 29484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd: No bootstrap required, opened a new log
I20260808 07:14:54.614370 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:54.609323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:57549 (local address 127.0.0.1:37861)
0808 07:14:54.609497 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:54.609504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:54.609528 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:14:54.609939 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:54.609945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:54.609991 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:54.610103 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:54.610355 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:54.611324 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:54.612397 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:54.612820 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:54.612906 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:54.613579 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:54.613583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:54.613583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:54.613735 (+ 152us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:54.613763 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:54.613774 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:54.613839 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:54.613960 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 07:14:54.614002 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260808 07:14:54.614502 29484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e05600cc38440c2aecc865a0730e0cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37861 } }
I20260808 07:14:54.614693 29484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:54.614744 29484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e05600cc38440c2aecc865a0730e0cd, State: Initialized, Role: FOLLOWER
I20260808 07:14:54.614892 29484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e05600cc38440c2aecc865a0730e0cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37861 } }
I20260808 07:14:54.614988 29484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:54.615039 29484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:54.615096 29484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:54.615330 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:54.615727 29484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e05600cc38440c2aecc865a0730e0cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37861 } }
I20260808 07:14:54.615850 29484 leader_election.cc:302] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e05600cc38440c2aecc865a0730e0cd; no voters:
I20260808 07:14:54.616014 29484 leader_election.cc:288] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:54.616067 29493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:54.616223 29493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [term 1 LEADER]: Becoming Leader. State: Replica: 8e05600cc38440c2aecc865a0730e0cd, State: Running, Role: LEADER
I20260808 07:14:54.616266 29484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:54.616400 29493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e05600cc38440c2aecc865a0730e0cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37861 } }
I20260808 07:14:54.616802 29493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e05600cc38440c2aecc865a0730e0cd. Latest consensus state: current_term: 1 leader_uuid: "8e05600cc38440c2aecc865a0730e0cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e05600cc38440c2aecc865a0730e0cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37861 } } }
I20260808 07:14:54.616813 29495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e05600cc38440c2aecc865a0730e0cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e05600cc38440c2aecc865a0730e0cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37861 } } }
I20260808 07:14:54.616950 29493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:54.616976 29495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:54.617434 29513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:54.617940 29513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:54.619695 29513 catalog_manager.cc:1347] Generated new cluster ID: 7c38a35eb0464460aea58d242b7c5fc1
I20260808 07:14:54.619781 29513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:54.620395 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:54.616650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:37545 (local address 127.0.0.1:37861)
0808 07:14:54.616689 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:54.616691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:54.616979 (+ 288us) server_negotiation.cc:407] Connection header received
0808 07:14:54.617083 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:54.617084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:54.617093 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:54.617121 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:54.617341 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:54.618027 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:54.619085 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:54.619380 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:54.619479 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:54.620147 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:54.620148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:54.620149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:54.620153 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:54.620166 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:54.620170 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:54.620223 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:54.620283 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:14:54.620291 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:14:54.625324 29513 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:54.625988 29513 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:54.636788 29513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e05600cc38440c2aecc865a0730e0cd: Generated new TSK 0
I20260808 07:14:54.636972 29513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:54.644524 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 29389
W20260808 07:14:54.660053 29367 connection.cc:535] client connection to 127.0.0.1:37861 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37861): Connection reset by peer (error 104)
2026-08-08T07:14:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:54 dist-test-slave-znh6 krb5kdc[29532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:54 dist-test-slave-znh6 krb5kdc[29532](info): set up 2 sockets
Aug 08 07:14:54 dist-test-slave-znh6 krb5kdc[29532](info): commencing operation
krb5kdc: starting...
W20260808 07:14:55.872460 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:55 dist-test-slave-znh6 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:55.993933 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:56.087771 29550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:56.088009 29550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:56.088056 29550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:56.089190 29550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:56.089232 29550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:56.089247 29550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:56.089286 29550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:56.089315 29550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:56.090744 29550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:56.091005 29550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:56.091223 29550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:56.096160 29561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:56.096205 29558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:56.096271 29550 server_base.cc:1034] running on GCE node
W20260808 07:14:56.096201 29567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:56.096544 29550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:56.096807 29550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:56.097961 29550 hybrid_clock.cc:648] HybridClock initialized: now 1786173296097952 us; error 30 us; skew 500 ppm
Aug 08 07:14:56 dist-test-slave-znh6 krb5kdc[29532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173296, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:56.100428 29550 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:56.100890 29550 webserver.cc:460] Webserver started at http://127.0.0.1:33981/ using document root <none> and password file <none>
I20260808 07:14:56.101130 29550 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:56.101197 29550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:56.101346 29550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:56.102662 29550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "9ffad8e998204abd80a585a933805993"
format_stamp: "Formatted at 2026-08-08 07:14:56 on dist-test-slave-znh6"
server_key: "3d2a3440904de12f14289a8ce79bcb0b"
server_key_iv: "82c7afbabc070beeb0df713ccfcc2095"
server_key_version: "encryptionkey@0"
I20260808 07:14:56.103070 29550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "9ffad8e998204abd80a585a933805993"
format_stamp: "Formatted at 2026-08-08 07:14:56 on dist-test-slave-znh6"
server_key: "3d2a3440904de12f14289a8ce79bcb0b"
server_key_iv: "82c7afbabc070beeb0df713ccfcc2095"
server_key_version: "encryptionkey@0"
I20260808 07:14:56.104933 29550 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:14:56.105703 29580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:56.105867 29550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:14:56.105973 29550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "9ffad8e998204abd80a585a933805993"
format_stamp: "Formatted at 2026-08-08 07:14:56 on dist-test-slave-znh6"
server_key: "3d2a3440904de12f14289a8ce79bcb0b"
server_key_iv: "82c7afbabc070beeb0df713ccfcc2095"
server_key_version: "encryptionkey@0"
I20260808 07:14:56.106056 29550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:56.109315 29550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:56.111176 29550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:56.111325 29550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:56.115372 29550 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37691
I20260808 07:14:56.115404 29649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37691 every 8 connection(s)
I20260808 07:14:56.115724 29550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:56.116665 29651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:56.118500 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 29550
I20260808 07:14:56.118625 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:56.118865 30937 external_mini_cluster.cc:1428] Setting key 17001e6aba67cb053e02b0a6cdb1e121
I20260808 07:14:56.119274 29651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993: Bootstrap starting.
I20260808 07:14:56.119939 29651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:56.120289 29651 log.cc:824] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:56.121131 29651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993: No bootstrap required, opened a new log
I20260808 07:14:56.121322 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:56.119364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:46541 (local address 127.0.0.1:37691)
0808 07:14:56.119533 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:56.119541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:56.119574 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:14:56.119971 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:56.119996 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:56.120030 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:56.120152 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:56.120855 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:56.120858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:56.120858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:56.120862 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:56.120898 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:56.120909 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:56.120970 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:56.121047 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:14:56.121097 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260808 07:14:56.122291 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:56.123896 29651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffad8e998204abd80a585a933805993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37691 } }
I20260808 07:14:56.124085 29651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:56.124142 29651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffad8e998204abd80a585a933805993, State: Initialized, Role: FOLLOWER
I20260808 07:14:56.124277 29651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffad8e998204abd80a585a933805993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37691 } }
I20260808 07:14:56.124382 29651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:56.124428 29651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:56.124485 29651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:56.125078 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:56.123972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:58521 (local address 127.0.0.1:37691)
0808 07:14:56.124012 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:56.124014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:56.124147 (+ 133us) server_negotiation.cc:407] Connection header received
0808 07:14:56.124260 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:56.124261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:56.124267 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:56.124329 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:56.124867 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:56.124868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:56.124868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:56.124871 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:56.124884 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:56.124887 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:56.124931 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:56.124992 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 07:14:56.125002 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:14:56.125226 29651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffad8e998204abd80a585a933805993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37691 } }
I20260808 07:14:56.125347 29651 leader_election.cc:302] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ffad8e998204abd80a585a933805993; no voters:
I20260808 07:14:56.125509 29651 leader_election.cc:288] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:56.125541 29664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:56.125699 29664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [term 1 LEADER]: Becoming Leader. State: Replica: 9ffad8e998204abd80a585a933805993, State: Running, Role: LEADER
I20260808 07:14:56.125763 29651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:56.125828 29664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffad8e998204abd80a585a933805993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37691 } }
I20260808 07:14:56.126187 29666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ffad8e998204abd80a585a933805993. Latest consensus state: current_term: 1 leader_uuid: "9ffad8e998204abd80a585a933805993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffad8e998204abd80a585a933805993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37691 } } }
I20260808 07:14:56.126181 29665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ffad8e998204abd80a585a933805993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffad8e998204abd80a585a933805993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37691 } } }
I20260808 07:14:56.126292 29666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:56.126297 29665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:56.126521 29676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:56.127034 29676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:56.128487 29676 catalog_manager.cc:1347] Generated new cluster ID: c02c259bf9b246a0a4ace28affc76850
I20260808 07:14:56.128537 29676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:56.133541 29676 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:56.133688 29676 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:56.142591 29676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ffad8e998204abd80a585a933805993: Generated new TSK 0
I20260808 07:14:56.142761 29676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:56.150942 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 29550
2026-08-08T07:14:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:56 dist-test-slave-znh6 krb5kdc[29691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:56 dist-test-slave-znh6 krb5kdc[29691](info): set up 2 sockets
Aug 08 07:14:56 dist-test-slave-znh6 krb5kdc[29691](info): commencing operation
krb5kdc: starting...
W20260808 07:14:57.357065 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:57 dist-test-slave-znh6 krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:57.478276 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44913
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:57.558049 29710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:57.558295 29710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:57.558334 29710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:57.559494 29710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:57.559568 29710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:57.559592 29710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:57.559612 29710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:57.559644 29710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:57.561112 29710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44913
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:57.561384 29710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:57.561614 29710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:57.566709 29716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:57.566704 29722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:57.566766 29710 server_base.cc:1034] running on GCE node
W20260808 07:14:57.566720 29717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:57.567202 29710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:57.567528 29710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:57.568691 29710 hybrid_clock.cc:648] HybridClock initialized: now 1786173297568691 us; error 37 us; skew 500 ppm
Aug 08 07:14:57 dist-test-slave-znh6 krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173297, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:57.571770 29710 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:57.572263 29710 webserver.cc:460] Webserver started at http://127.0.0.1:37637/ using document root <none> and password file <none>
I20260808 07:14:57.572489 29710 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:57.572561 29710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:57.572695 29710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:57.573719 29710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "b77bd4575a4a4135a017de2bba013d01"
format_stamp: "Formatted at 2026-08-08 07:14:57 on dist-test-slave-znh6"
server_key: "14ae6ca8cd6920d28fac71bdaebe498c"
server_key_iv: "9ddd9ea08c64ad6b656e3d27a1841a5d"
server_key_version: "encryptionkey@0"
I20260808 07:14:57.574090 29710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "b77bd4575a4a4135a017de2bba013d01"
format_stamp: "Formatted at 2026-08-08 07:14:57 on dist-test-slave-znh6"
server_key: "14ae6ca8cd6920d28fac71bdaebe498c"
server_key_iv: "9ddd9ea08c64ad6b656e3d27a1841a5d"
server_key_version: "encryptionkey@0"
I20260808 07:14:57.575536 29710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:14:57.576365 29735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:57.576649 29710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:14:57.576752 29710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "b77bd4575a4a4135a017de2bba013d01"
format_stamp: "Formatted at 2026-08-08 07:14:57 on dist-test-slave-znh6"
server_key: "14ae6ca8cd6920d28fac71bdaebe498c"
server_key_iv: "9ddd9ea08c64ad6b656e3d27a1841a5d"
server_key_version: "encryptionkey@0"
I20260808 07:14:57.576830 29710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:57.597054 29710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:57.599368 29710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:57.599536 29710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:57.603683 29710 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44913
I20260808 07:14:57.603720 29807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44913 every 8 connection(s)
I20260808 07:14:57.604096 29710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:57.604938 29808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:57.607347 29808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01: Bootstrap starting.
I20260808 07:14:57.607882 29808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:57.608125 29808 log.cc:824] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:57.608763 29808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01: No bootstrap required, opened a new log
I20260808 07:14:57.610291 29808 raft_consensus.cc:348] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77bd4575a4a4135a017de2bba013d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } }
I20260808 07:14:57.610430 29808 raft_consensus.cc:374] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:57.610452 29808 raft_consensus.cc:729] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b77bd4575a4a4135a017de2bba013d01, State: Initialized, Role: FOLLOWER
I20260808 07:14:57.610550 29808 consensus_queue.cc:260] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77bd4575a4a4135a017de2bba013d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } }
I20260808 07:14:57.610596 29808 raft_consensus.cc:388] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:57.610616 29808 raft_consensus.cc:482] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:57.610646 29808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:57.611214 29808 raft_consensus.cc:504] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77bd4575a4a4135a017de2bba013d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } }
I20260808 07:14:57.611300 29808 leader_election.cc:302] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b77bd4575a4a4135a017de2bba013d01; no voters:
I20260808 07:14:57.611423 29808 leader_election.cc:288] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:57.611500 29811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:57.611657 29811 raft_consensus.cc:686] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [term 1 LEADER]: Becoming Leader. State: Replica: b77bd4575a4a4135a017de2bba013d01, State: Running, Role: LEADER
I20260808 07:14:57.611687 29808 sys_catalog.cc:564] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:57.611747 29811 consensus_queue.cc:237] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77bd4575a4a4135a017de2bba013d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } }
I20260808 07:14:57.612146 29814 sys_catalog.cc:455] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b77bd4575a4a4135a017de2bba013d01. Latest consensus state: current_term: 1 leader_uuid: "b77bd4575a4a4135a017de2bba013d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77bd4575a4a4135a017de2bba013d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } } }
I20260808 07:14:57.612231 29814 sys_catalog.cc:458] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:57.612382 29812 sys_catalog.cc:455] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b77bd4575a4a4135a017de2bba013d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77bd4575a4a4135a017de2bba013d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } } }
I20260808 07:14:57.612524 29812 sys_catalog.cc:458] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01 [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:57.612636 29824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:57.613137 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 29710
I20260808 07:14:57.613173 29824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:57.613273 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:57.613539 30937 external_mini_cluster.cc:1428] Setting key 3e844682e7430af8a5865b97849463a6
I20260808 07:14:57.614926 29824 catalog_manager.cc:1347] Generated new cluster ID: 0f24e57b4b2b46d19fd24a17255bb4ac
I20260808 07:14:57.614989 29824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:14:57 dist-test-slave-znh6 krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:14:57.623435 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:57.614183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:48281 (local address 127.0.0.1:44913)
0808 07:14:57.614354 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:57.614362 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:57.614391 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:14:57.614502 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:57.614508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:57.614564 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:57.614702 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:57.615451 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:57.616359 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:57.617125 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:57.617575 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:57.617693 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:57.620157 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:57.620161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:57.620161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:57.620172 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:57.621806 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:57.622216 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:57.622219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:57.622220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:57.622323 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:57.622570 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:57.622571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:57.622571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:57.622795 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:57.623031 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:57.623116 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:14:57.623221 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260808 07:14:57.624276 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:57.632869 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:57.625584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:48325 (local address 127.0.0.1:44913)
0808 07:14:57.625620 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:57.625622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:57.626986 (+ 1364us) server_negotiation.cc:407] Connection header received
0808 07:14:57.627048 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:57.627049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:57.627060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:57.627103 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:57.627604 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:57.628080 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:57.628760 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:57.628992 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:57.629066 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:57.630232 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:57.630234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:57.630234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:14:57.630242 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:57.631697 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:57.632071 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:57.632073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:57.632074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:57.632127 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:14:57.632364 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:14:57.632365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:14:57.632365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:14:57.632472 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:57.632578 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:57.632701 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 07:14:57.632763 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:14:57.640388 29824 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:57.641083 29824 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:57.648229 29824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b77bd4575a4a4135a017de2bba013d01: Generated new TSK 0
I20260808 07:14:57.648397 29824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:57.652509 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 29710
W20260808 07:14:57.667289 29689 connection.cc:535] client connection to 127.0.0.1:44913 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44913): Connection reset by peer (error 104)
2026-08-08T07:14:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:57 dist-test-slave-znh6 krb5kdc[29852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:57 dist-test-slave-znh6 krb5kdc[29852](info): set up 2 sockets
Aug 08 07:14:57 dist-test-slave-znh6 krb5kdc[29852](info): commencing operation
krb5kdc: starting...
W20260808 07:14:58.878577 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:14:58 dist-test-slave-znh6 krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:14:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:14:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:14:58.993804 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:14:59.073297 29870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:14:59.073487 29870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:14:59.073509 29870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:14:59.074635 29870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:14:59.074676 29870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:14:59.074692 29870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:14:59.074741 29870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:14:59.074761 29870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:14:59.076274 29870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38173
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:14:59.076519 29870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:14:59.076737 29870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:14:59.081624 29880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:14:59.081635 29878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:59.081664 29870 server_base.cc:1034] running on GCE node
W20260808 07:14:59.081635 29877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:14:59.082094 29870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:14:59.082368 29870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:14:59.083511 29870 hybrid_clock.cc:648] HybridClock initialized: now 1786173299083497 us; error 42 us; skew 500 ppm
Aug 08 07:14:59 dist-test-slave-znh6 krb5kdc[29852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173299, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:14:59.085876 29870 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:14:59.086302 29870 webserver.cc:460] Webserver started at http://127.0.0.1:44411/ using document root <none> and password file <none>
I20260808 07:14:59.086480 29870 fs_manager.cc:359] Metadata directory not provided
I20260808 07:14:59.086519 29870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:14:59.086602 29870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:14:59.087599 29870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "3b292f63dd77457a8a4823b4baa6f47d"
format_stamp: "Formatted at 2026-08-08 07:14:59 on dist-test-slave-znh6"
server_key: "0c6c3895fad2d87e229c3213f01b74d5"
server_key_iv: "e23d1c4ebc6350643a24582d1b3320af"
server_key_version: "encryptionkey@0"
I20260808 07:14:59.087890 29870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "3b292f63dd77457a8a4823b4baa6f47d"
format_stamp: "Formatted at 2026-08-08 07:14:59 on dist-test-slave-znh6"
server_key: "0c6c3895fad2d87e229c3213f01b74d5"
server_key_iv: "e23d1c4ebc6350643a24582d1b3320af"
server_key_version: "encryptionkey@0"
I20260808 07:14:59.089210 29870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:14:59.090210 29888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:14:59.090395 29870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:14:59.090487 29870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "3b292f63dd77457a8a4823b4baa6f47d"
format_stamp: "Formatted at 2026-08-08 07:14:59 on dist-test-slave-znh6"
server_key: "0c6c3895fad2d87e229c3213f01b74d5"
server_key_iv: "e23d1c4ebc6350643a24582d1b3320af"
server_key_version: "encryptionkey@0"
I20260808 07:14:59.090559 29870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:14:59.100509 29870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:14:59.102118 29870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:14:59.102268 29870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:14:59.106387 29870 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38173
I20260808 07:14:59.106436 29964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38173 every 8 connection(s)
I20260808 07:14:59.106760 29870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:14:59.107625 29966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:14:59.108218 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 29870
I20260808 07:14:59.108342 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:14:59.108546 30937 external_mini_cluster.cc:1428] Setting key 264612bfd0f8f25408b61839da315eff
I20260808 07:14:59.110508 29966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d: Bootstrap starting.
I20260808 07:14:59.111219 29966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d: Neither blocks nor log segments found. Creating new log.
I20260808 07:14:59.111528 29966 log.cc:824] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:14:59.112262 29966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d: No bootstrap required, opened a new log
I20260808 07:14:59.114092 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:59.109037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:33721 (local address 127.0.0.1:38173)
0808 07:14:59.109210 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:59.109217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:59.109239 (+ 22us) server_negotiation.cc:407] Connection header received
0808 07:14:59.109706 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:59.109713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:59.109759 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:59.109866 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:59.110118 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:59.111073 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:59.112136 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:59.112569 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:59.112647 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:59.113317 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:59.113321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:59.113322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:59.113526 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:59.113562 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:59.113577 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:59.113660 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:59.113847 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 07:14:59.113875 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260808 07:14:59.115010 29966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b292f63dd77457a8a4823b4baa6f47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38173 } }
I20260808 07:14:59.115196 29966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:14:59.115247 29966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b292f63dd77457a8a4823b4baa6f47d, State: Initialized, Role: FOLLOWER
I20260808 07:14:59.115366 29966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b292f63dd77457a8a4823b4baa6f47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38173 } }
I20260808 07:14:59.115448 29966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:14:59.115480 29966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:14:59.115520 29966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:14:59.116170 29966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b292f63dd77457a8a4823b4baa6f47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38173 } }
I20260808 07:14:59.116283 29966 leader_election.cc:302] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b292f63dd77457a8a4823b4baa6f47d; no voters:
I20260808 07:14:59.116447 29966 leader_election.cc:288] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:14:59.116494 29972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:14:59.116622 29972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [term 1 LEADER]: Becoming Leader. State: Replica: 3b292f63dd77457a8a4823b4baa6f47d, State: Running, Role: LEADER
I20260808 07:14:59.116698 29966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:14:59.116700 29972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b292f63dd77457a8a4823b4baa6f47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38173 } }
I20260808 07:14:59.116926 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:14:59.117018 29972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b292f63dd77457a8a4823b4baa6f47d. Latest consensus state: current_term: 1 leader_uuid: "3b292f63dd77457a8a4823b4baa6f47d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b292f63dd77457a8a4823b4baa6f47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38173 } } }
I20260808 07:14:59.117095 29972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:59.117056 29974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b292f63dd77457a8a4823b4baa6f47d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b292f63dd77457a8a4823b4baa6f47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38173 } } }
I20260808 07:14:59.117241 29974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d [sys.catalog]: This master's current role is: LEADER
I20260808 07:14:59.117308 29979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:14:59.117839 29979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:14:59.119515 29979 catalog_manager.cc:1347] Generated new cluster ID: e4d0e52ddfd3427ea9f5f2e982afd01d
I20260808 07:14:59.119572 29979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:14:59.122853 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:14:59.119865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:44877 (local address 127.0.0.1:38173)
0808 07:14:59.119898 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:14:59.119900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:14:59.120034 (+ 134us) server_negotiation.cc:407] Connection header received
0808 07:14:59.120106 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:14:59.120107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:14:59.120117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:14:59.120189 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:14:59.120350 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:59.120811 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:59.121532 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:14:59.121796 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:14:59.121862 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:14:59.122592 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:14:59.122594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:14:59.122594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:14:59.122598 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:14:59.122611 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:14:59.122614 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:14:59.122668 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:14:59.122729 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 07:14:59.122738 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:14:59.148167 29979 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:14:59.148661 29979 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:14:59.150913 29979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b292f63dd77457a8a4823b4baa6f47d: Generated new TSK 0
I20260808 07:14:59.151099 29979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:14:59.156354 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 29870
W20260808 07:14:59.168447 29850 connection.cc:535] client connection to 127.0.0.1:38173 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38173): Connection reset by peer (error 104)
2026-08-08T07:14:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:14:59 dist-test-slave-znh6 krb5kdc[30004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:14:59 dist-test-slave-znh6 krb5kdc[30004](info): set up 2 sockets
Aug 08 07:14:59 dist-test-slave-znh6 krb5kdc[30004](info): commencing operation
krb5kdc: starting...
W20260808 07:15:00.369809 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:15:00 dist-test-slave-znh6 krb5kdc[30004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:15:00.493991 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:00.575783 30021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:00.576035 30021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:00.576074 30021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:00.577215 30021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:00.577292 30021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:00.577324 30021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:00.577353 30021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:00.577376 30021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:00.578867 30021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37561
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:00.579133 30021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:00.579382 30021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:00.584477 30028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:00.584518 30021 server_base.cc:1034] running on GCE node
W20260808 07:15:00.584506 30030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:00.584487 30033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:00.585011 30021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:00.585309 30021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:00.586453 30021 hybrid_clock.cc:648] HybridClock initialized: now 1786173300586431 us; error 35 us; skew 500 ppm
Aug 08 07:15:00 dist-test-slave-znh6 krb5kdc[30004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173300, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:00.589015 30021 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:15:00.589493 30021 webserver.cc:460] Webserver started at http://127.0.0.1:40405/ using document root <none> and password file <none>
I20260808 07:15:00.589731 30021 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:00.589802 30021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:00.589951 30021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:00.591044 30021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "91c4c74e80a94c359263997e4f7cde0d"
format_stamp: "Formatted at 2026-08-08 07:15:00 on dist-test-slave-znh6"
server_key: "2d1eb5bc4a318c2d20a4e204d4e82a49"
server_key_iv: "6e4f4e92989313fc0fe1f2288ec8bb86"
server_key_version: "encryptionkey@0"
I20260808 07:15:00.591399 30021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "91c4c74e80a94c359263997e4f7cde0d"
format_stamp: "Formatted at 2026-08-08 07:15:00 on dist-test-slave-znh6"
server_key: "2d1eb5bc4a318c2d20a4e204d4e82a49"
server_key_iv: "6e4f4e92989313fc0fe1f2288ec8bb86"
server_key_version: "encryptionkey@0"
I20260808 07:15:00.592733 30021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:00.593479 30044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:00.593669 30021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:00.593776 30021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "91c4c74e80a94c359263997e4f7cde0d"
format_stamp: "Formatted at 2026-08-08 07:15:00 on dist-test-slave-znh6"
server_key: "2d1eb5bc4a318c2d20a4e204d4e82a49"
server_key_iv: "6e4f4e92989313fc0fe1f2288ec8bb86"
server_key_version: "encryptionkey@0"
I20260808 07:15:00.593848 30021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:00.597151 30021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:00.599385 30021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:00.599555 30021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:00.603583 30021 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37561
I20260808 07:15:00.603637 30111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37561 every 8 connection(s)
I20260808 07:15:00.603962 30021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:00.604920 30113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:00.607163 30113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d: Bootstrap starting.
I20260808 07:15:00.607723 30113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:00.608006 30113 log.cc:824] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:00.608345 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 30021
I20260808 07:15:00.608462 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:15:00.608712 30937 external_mini_cluster.cc:1428] Setting key 07349f96601ba6070a8ec82efec20063
I20260808 07:15:00.608819 30113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d: No bootstrap required, opened a new log
I20260808 07:15:00.610703 30113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c4c74e80a94c359263997e4f7cde0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } }
I20260808 07:15:00.610877 30113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:00.610930 30113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c4c74e80a94c359263997e4f7cde0d, State: Initialized, Role: FOLLOWER
I20260808 07:15:00.611059 30113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c4c74e80a94c359263997e4f7cde0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } }
I20260808 07:15:00.611155 30113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:00.611193 30113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:00.611248 30113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:00.611351 30118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:00.609193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:35183 (local address 127.0.0.1:37561)
0808 07:15:00.609387 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:00.609396 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:00.609432 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:15:00.610053 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:00.610060 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:00.610086 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:00.610230 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:00.610846 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:00.610849 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:00.610850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:00.610854 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:00.610893 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:00.610907 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:00.610967 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:00.611086 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 07:15:00.611131 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260808 07:15:00.612185 30113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c4c74e80a94c359263997e4f7cde0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } }
I20260808 07:15:00.612291 30113 leader_election.cc:302] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91c4c74e80a94c359263997e4f7cde0d; no voters:
I20260808 07:15:00.612484 30113 leader_election.cc:288] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:00.612520 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:00.612639 30119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:00.612777 30113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:00.612874 30119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [term 1 LEADER]: Becoming Leader. State: Replica: 91c4c74e80a94c359263997e4f7cde0d, State: Running, Role: LEADER
I20260808 07:15:00.613008 30119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c4c74e80a94c359263997e4f7cde0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } }
I20260808 07:15:00.613533 30120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91c4c74e80a94c359263997e4f7cde0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c4c74e80a94c359263997e4f7cde0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } } }
I20260808 07:15:00.613809 30120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:00.613504 30121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91c4c74e80a94c359263997e4f7cde0d. Latest consensus state: current_term: 1 leader_uuid: "91c4c74e80a94c359263997e4f7cde0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c4c74e80a94c359263997e4f7cde0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } } }
I20260808 07:15:00.613953 30121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:00.614315 30139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:00.614894 30139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:00.616266 30118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:00.614687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.140:55507 (local address 127.0.0.1:37561)
0808 07:15:00.614725 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:00.614727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:00.615297 (+ 570us) server_negotiation.cc:407] Connection header received
0808 07:15:00.615381 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:00.615382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:00.615387 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:00.615443 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:00.616055 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:00.616056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:00.616056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:00.616061 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:00.616073 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:00.616077 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:00.616133 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:00.616190 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:15:00.616204 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:00.619244 30139 catalog_manager.cc:1347] Generated new cluster ID: 5ea8e54bf9fe4d75aaf09d9cfbefbae2
I20260808 07:15:00.619287 30139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:00.631178 30139 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:00.631346 30139 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:00.641328 30139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91c4c74e80a94c359263997e4f7cde0d: Generated new TSK 0
I20260808 07:15:00.641506 30139 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:00.647617 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 30021
2026-08-08T07:15:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:00 dist-test-slave-znh6 krb5kdc[30153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:00 dist-test-slave-znh6 krb5kdc[30153](info): set up 2 sockets
Aug 08 07:15:00 dist-test-slave-znh6 krb5kdc[30153](info): commencing operation
krb5kdc: starting...
W20260808 07:15:01.879678 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:15:01 dist-test-slave-znh6 krb5kdc[30153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:15:01.998224 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41241
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:45513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:41241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:02.079499 30171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:02.079694 30171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:02.079715 30171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:02.080751 30171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:02.080783 30171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:02.080796 30171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:02.080809 30171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:02.080821 30171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:02.082180 30171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:45513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:41241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:02.082391 30171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:02.082597 30171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:02.087452 30179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:02.087541 30171 server_base.cc:1034] running on GCE node
W20260808 07:15:02.087452 30181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:02.087522 30178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:02.087966 30171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:02.088274 30171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:02.089463 30171 hybrid_clock.cc:648] HybridClock initialized: now 1786173302089449 us; error 27 us; skew 500 ppm
Aug 08 07:15:02 dist-test-slave-znh6 krb5kdc[30153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173302, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:02.098520 30171 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:15:02.099005 30171 webserver.cc:460] Webserver started at http://127.30.54.126:40871/ using document root <none> and password file <none>
I20260808 07:15:02.099206 30171 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:02.099246 30171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:02.099332 30171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:02.100318 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "3c13e52f25f34867b980b474a62a2865"
format_stamp: "Formatted at 2026-08-08 07:15:02 on dist-test-slave-znh6"
server_key: "c4519e036cc50b497d71de81f3073983"
server_key_iv: "47776b7f187b7bec4574f75089dba98a"
server_key_version: "encryptionkey@0"
I20260808 07:15:02.100620 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "3c13e52f25f34867b980b474a62a2865"
format_stamp: "Formatted at 2026-08-08 07:15:02 on dist-test-slave-znh6"
server_key: "c4519e036cc50b497d71de81f3073983"
server_key_iv: "47776b7f187b7bec4574f75089dba98a"
server_key_version: "encryptionkey@0"
I20260808 07:15:02.101851 30171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:02.102532 30194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:02.102732 30171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:02.102869 30171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "3c13e52f25f34867b980b474a62a2865"
format_stamp: "Formatted at 2026-08-08 07:15:02 on dist-test-slave-znh6"
server_key: "c4519e036cc50b497d71de81f3073983"
server_key_iv: "47776b7f187b7bec4574f75089dba98a"
server_key_version: "encryptionkey@0"
I20260808 07:15:02.102950 30171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:02.120358 30171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:02.122148 30171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:02.122305 30171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:02.127321 30171 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:41241
I20260808 07:15:02.127353 30269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:41241 every 8 connection(s)
I20260808 07:15:02.127835 30171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:02.128883 30270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:02.131173 30270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865: Bootstrap starting.
I20260808 07:15:02.131713 30270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:02.131946 30270 log.cc:824] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:02.132638 30270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865: No bootstrap required, opened a new log
I20260808 07:15:02.133121 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 30171
I20260808 07:15:02.133258 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:15:02.133452 30937 external_mini_cluster.cc:1428] Setting key ee7bb42946ef2163575bf4abd92d13a9
I20260808 07:15:02.135296 30270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c13e52f25f34867b980b474a62a2865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41241 } }
I20260808 07:15:02.135478 30270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:02.135520 30270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c13e52f25f34867b980b474a62a2865, State: Initialized, Role: FOLLOWER
I20260808 07:15:02.135666 30270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c13e52f25f34867b980b474a62a2865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41241 } }
I20260808 07:15:02.135748 30270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:02.135771 30270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:02.135841 30270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:02.136488 30270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c13e52f25f34867b980b474a62a2865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41241 } }
I20260808 07:15:02.136595 30270 leader_election.cc:302] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c13e52f25f34867b980b474a62a2865; no voters:
I20260808 07:15:02.136755 30270 leader_election.cc:288] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:02.136821 30277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:02.136992 30270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:02.137017 30277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [term 1 LEADER]: Becoming Leader. State: Replica: 3c13e52f25f34867b980b474a62a2865, State: Running, Role: LEADER
I20260808 07:15:02.137101 30277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c13e52f25f34867b980b474a62a2865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41241 } }
I20260808 07:15:02.137466 30277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c13e52f25f34867b980b474a62a2865. Latest consensus state: current_term: 1 leader_uuid: "3c13e52f25f34867b980b474a62a2865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c13e52f25f34867b980b474a62a2865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41241 } } }
I20260808 07:15:02.137564 30277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:02.137480 30278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c13e52f25f34867b980b474a62a2865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c13e52f25f34867b980b474a62a2865" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41241 } } }
I20260808 07:15:02.137724 30278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865 [sys.catalog]: This master's current role is: LEADER
W20260808 07:15:02.138799 30294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:02.139087 30294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:02.139254 30287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:02.139456 30287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:02.140936 30287 catalog_manager.cc:1347] Generated new cluster ID: 7f95bf22e8d248c5aec46e0bacde78dd
I20260808 07:15:02.140998 30287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:15:02 dist-test-slave-znh6 krb5kdc[30153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:15:02.147055 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:02.134014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46979 (local address 127.30.54.126:41241)
0808 07:15:02.134210 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:02.134222 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:02.134257 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:15:02.135245 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:02.135255 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:02.135318 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:02.135471 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:02.136243 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:02.137642 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:02.138974 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:02.139598 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:02.139702 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:02.142691 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:02.142695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:02.142696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:02.142706 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:02.145288 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:02.145754 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:02.145757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:02.145757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:02.145856 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:02.146117 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:02.146117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:02.146118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:02.146396 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:02.146542 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:02.146715 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 07:15:02.146834 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260808 07:15:02.148260 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:02.153259 30287 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:02.154016 30287 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:02.156992 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:02.149648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35427 (local address 127.30.54.126:41241)
0808 07:15:02.149678 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:02.149679 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:02.149822 (+ 143us) server_negotiation.cc:407] Connection header received
0808 07:15:02.149905 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:02.149906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:02.149915 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:02.149942 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:02.150593 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:02.151373 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:02.152354 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:02.152674 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:02.152716 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:02.154315 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:02.154316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:02.154316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:02.154322 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:02.155829 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:02.156210 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:02.156210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:02.156211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:02.156256 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:02.156465 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:02.156466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:02.156466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:02.156571 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:02.156636 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:02.156850 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 07:15:02.156906 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 07:15:02.164676 30287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c13e52f25f34867b980b474a62a2865: Generated new TSK 0
I20260808 07:15:02.164894 30287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:02.167614 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 30171
W20260808 07:15:02.181753 30151 connection.cc:535] client connection to 127.30.54.126:41241 recv error: Network error: failed to read from TLS socket (remote: 127.30.54.126:41241): Connection reset by peer (error 104)
2026-08-08T07:15:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1517 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:02 dist-test-slave-znh6 krb5kdc[30311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:02 dist-test-slave-znh6 krb5kdc[30311](info): set up 2 sockets
Aug 08 07:15:02 dist-test-slave-znh6 krb5kdc[30311](info): commencing operation
krb5kdc: starting...
W20260808 07:15:03.394260 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:15:03 dist-test-slave-znh6 krb5kdc[30311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:15:03.521502 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:44069
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:44069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:03.602699 30328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:03.602912 30328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:03.602944 30328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:03.604107 30328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:03.604149 30328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:03.604163 30328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:03.604177 30328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:03.604189 30328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:03.605657 30328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:44069
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:44069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:03.605914 30328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:03.606149 30328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:03.611155 30336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:03.611130 30337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:03.611194 30328 server_base.cc:1034] running on GCE node
W20260808 07:15:03.611153 30340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:03.611662 30328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:03.611985 30328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:03.613122 30328 hybrid_clock.cc:648] HybridClock initialized: now 1786173303613109 us; error 29 us; skew 500 ppm
Aug 08 07:15:03 dist-test-slave-znh6 krb5kdc[30311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173303, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:03.615684 30328 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:15:03.616128 30328 webserver.cc:460] Webserver started at http://127.30.54.126:36925/ using document root <none> and password file <none>
I20260808 07:15:03.616351 30328 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:03.616425 30328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:03.616585 30328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:03.617632 30328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "94983d0e3b64434186acacabe2820e37"
format_stamp: "Formatted at 2026-08-08 07:15:03 on dist-test-slave-znh6"
server_key: "ff598b1b691a6925bdb837ec556b44ba"
server_key_iv: "5399f5f86fabf7d102dd4d3b7d5cf59a"
server_key_version: "encryptionkey@0"
I20260808 07:15:03.618002 30328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "94983d0e3b64434186acacabe2820e37"
format_stamp: "Formatted at 2026-08-08 07:15:03 on dist-test-slave-znh6"
server_key: "ff598b1b691a6925bdb837ec556b44ba"
server_key_iv: "5399f5f86fabf7d102dd4d3b7d5cf59a"
server_key_version: "encryptionkey@0"
I20260808 07:15:03.619436 30328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:15:03.620302 30350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:03.620481 30328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:03.620590 30328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "94983d0e3b64434186acacabe2820e37"
format_stamp: "Formatted at 2026-08-08 07:15:03 on dist-test-slave-znh6"
server_key: "ff598b1b691a6925bdb837ec556b44ba"
server_key_iv: "5399f5f86fabf7d102dd4d3b7d5cf59a"
server_key_version: "encryptionkey@0"
I20260808 07:15:03.620676 30328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:03.631584 30328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:03.633386 30328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:03.633556 30328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:03.637741 30328 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:44069
I20260808 07:15:03.637777 30418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:44069 every 8 connection(s)
I20260808 07:15:03.638125 30328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:03.639096 30419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:03.641284 30419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37: Bootstrap starting.
I20260808 07:15:03.641803 30419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:03.642059 30419 log.cc:824] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:03.642727 30419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37: No bootstrap required, opened a new log
I20260808 07:15:03.644361 30419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94983d0e3b64434186acacabe2820e37" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44069 } }
I20260808 07:15:03.644500 30419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:03.644522 30419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94983d0e3b64434186acacabe2820e37, State: Initialized, Role: FOLLOWER
I20260808 07:15:03.644659 30419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94983d0e3b64434186acacabe2820e37" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44069 } }
I20260808 07:15:03.644743 30419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:03.644766 30419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:03.644795 30419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:03.645582 30419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94983d0e3b64434186acacabe2820e37" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44069 } }
I20260808 07:15:03.645669 30419 leader_election.cc:302] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94983d0e3b64434186acacabe2820e37; no voters:
I20260808 07:15:03.645821 30419 leader_election.cc:288] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:03.645855 30422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:03.645975 30422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [term 1 LEADER]: Becoming Leader. State: Replica: 94983d0e3b64434186acacabe2820e37, State: Running, Role: LEADER
I20260808 07:15:03.646104 30419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:03.646130 30422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94983d0e3b64434186acacabe2820e37" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44069 } }
I20260808 07:15:03.646862 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 30328
I20260808 07:15:03.646901 30424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94983d0e3b64434186acacabe2820e37. Latest consensus state: current_term: 1 leader_uuid: "94983d0e3b64434186acacabe2820e37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94983d0e3b64434186acacabe2820e37" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44069 } } }
I20260808 07:15:03.646977 30424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:03.646927 30423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94983d0e3b64434186acacabe2820e37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94983d0e3b64434186acacabe2820e37" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44069 } } }
I20260808 07:15:03.646989 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:15:03.647015 30423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:03.647221 30937 external_mini_cluster.cc:1428] Setting key d573a1314330430f97921dc67f416e90
I20260808 07:15:03.647229 30434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:03.647774 30434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:03.649371 30434 catalog_manager.cc:1347] Generated new cluster ID: 0c1b1777000f4e6e89d342d4eaac3a2c
I20260808 07:15:03.649427 30434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:03.652840 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:03.647707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36399 (local address 127.30.54.126:44069)
0808 07:15:03.647934 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:03.647944 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:03.648245 (+ 301us) server_negotiation.cc:407] Connection header received
0808 07:15:03.648388 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:03.648395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:03.648473 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:03.648630 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:03.648871 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:03.650024 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:03.650884 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:03.651337 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:03.651459 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:03.652182 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:03.652186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:03.652186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:03.652371 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:03.652406 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:03.652417 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:03.652513 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:03.652596 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 07:15:03.652624 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
I20260808 07:15:03.653771 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:03.658057 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:03.655163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34005 (local address 127.30.54.126:44069)
0808 07:15:03.655200 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:03.655202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:03.655336 (+ 134us) server_negotiation.cc:407] Connection header received
0808 07:15:03.655407 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:03.655408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:03.655420 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:03.655475 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:03.655633 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:03.656095 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:03.656811 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:03.657049 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:03.657127 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:03.657808 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:03.657809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:03.657809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:03.657814 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:03.657827 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:03.657831 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:03.657885 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:03.657946 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 07:15:03.657958 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:03.686429 30434 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:03.687114 30434 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:03.692085 30434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94983d0e3b64434186acacabe2820e37: Generated new TSK 0
I20260808 07:15:03.692294 30434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:03.694628 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 30328
W20260808 07:15:03.705041 30308 connection.cc:535] client connection to 127.30.54.126:44069 recv error: Network error: failed to read from TLS socket (remote: 127.30.54.126:44069): Connection reset by peer (error 104)
2026-08-08T07:15:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:03 dist-test-slave-znh6 krb5kdc[30458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:03 dist-test-slave-znh6 krb5kdc[30458](info): set up 2 sockets
Aug 08 07:15:03 dist-test-slave-znh6 krb5kdc[30458](info): commencing operation
krb5kdc: starting...
W20260808 07:15:04.917523 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:15:04 dist-test-slave-znh6 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:15:05.037647 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:44941
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:44941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:05.117766 30476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:05.118005 30476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:05.118042 30476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:05.119192 30476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:05.119264 30476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:05.119288 30476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:05.119305 30476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:05.119326 30476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:05.120743 30476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:44941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:44941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:05.121004 30476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:05.121236 30476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:05.126106 30485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:05.126252 30476 server_base.cc:1034] running on GCE node
W20260808 07:15:05.126119 30484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:05.126132 30488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:05.126649 30476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:05.126964 30476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:05.128115 30476 hybrid_clock.cc:648] HybridClock initialized: now 1786173305128103 us; error 30 us; skew 500 ppm
Aug 08 07:15:05 dist-test-slave-znh6 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:05.130673 30476 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:15:05.131186 30476 webserver.cc:460] Webserver started at http://127.30.54.126:34939/ using document root <none> and password file <none>
I20260808 07:15:05.131428 30476 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:05.131500 30476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:05.131654 30476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:05.132654 30476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "ffc65fb71dc04d12ac503f8bf5453583"
format_stamp: "Formatted at 2026-08-08 07:15:05 on dist-test-slave-znh6"
server_key: "12ea8c1bc189675629d92d3f9be6626b"
server_key_iv: "5e4e695ffcda3e9b567b282660512a00"
server_key_version: "encryptionkey@0"
I20260808 07:15:05.133020 30476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "ffc65fb71dc04d12ac503f8bf5453583"
format_stamp: "Formatted at 2026-08-08 07:15:05 on dist-test-slave-znh6"
server_key: "12ea8c1bc189675629d92d3f9be6626b"
server_key_iv: "5e4e695ffcda3e9b567b282660512a00"
server_key_version: "encryptionkey@0"
I20260808 07:15:05.134358 30476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:05.135197 30499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:05.135386 30476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:15:05.135488 30476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "ffc65fb71dc04d12ac503f8bf5453583"
format_stamp: "Formatted at 2026-08-08 07:15:05 on dist-test-slave-znh6"
server_key: "12ea8c1bc189675629d92d3f9be6626b"
server_key_iv: "5e4e695ffcda3e9b567b282660512a00"
server_key_version: "encryptionkey@0"
I20260808 07:15:05.135574 30476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:05.139986 30476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:05.141721 30476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:05.141872 30476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:05.145943 30476 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:44941
I20260808 07:15:05.145977 30570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:44941 every 8 connection(s)
I20260808 07:15:05.146291 30476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:05.147327 30572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:05.149507 30572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583: Bootstrap starting.
I20260808 07:15:05.150084 30572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:05.150381 30572 log.cc:824] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:05.151130 30572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583: No bootstrap required, opened a new log
I20260808 07:15:05.152014 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 30476
I20260808 07:15:05.152105 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:15:05.152330 30937 external_mini_cluster.cc:1428] Setting key 38c0a631eba34d7c03f30715b1cc4841
I20260808 07:15:05.153230 30572 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc65fb71dc04d12ac503f8bf5453583" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44941 } }
I20260808 07:15:05.153398 30572 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:05.153438 30572 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc65fb71dc04d12ac503f8bf5453583, State: Initialized, Role: FOLLOWER
I20260808 07:15:05.153599 30572 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc65fb71dc04d12ac503f8bf5453583" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44941 } }
I20260808 07:15:05.153704 30572 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:05.153750 30572 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:05.153798 30572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:05.154675 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:05.152843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41091 (local address 127.30.54.126:44941)
0808 07:15:05.153047 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:05.153058 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:05.153093 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:15:05.153207 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:05.153214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:05.153243 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:05.153392 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:05.154189 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:05.154193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:05.154194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:05.154198 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:05.154253 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:05.154272 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:05.154356 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:05.154416 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 07:15:05.154454 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":92,"threads_started":1}
I20260808 07:15:05.154884 30572 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc65fb71dc04d12ac503f8bf5453583" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44941 } }
I20260808 07:15:05.154997 30572 leader_election.cc:302] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffc65fb71dc04d12ac503f8bf5453583; no voters:
I20260808 07:15:05.155182 30572 leader_election.cc:288] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:05.155210 30580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:05.155318 30580 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [term 1 LEADER]: Becoming Leader. State: Replica: ffc65fb71dc04d12ac503f8bf5453583, State: Running, Role: LEADER
I20260808 07:15:05.155437 30572 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:05.155452 30580 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc65fb71dc04d12ac503f8bf5453583" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44941 } }
I20260808 07:15:05.155613 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:05.155870 30581 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffc65fb71dc04d12ac503f8bf5453583. Latest consensus state: current_term: 1 leader_uuid: "ffc65fb71dc04d12ac503f8bf5453583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc65fb71dc04d12ac503f8bf5453583" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44941 } } }
I20260808 07:15:05.155951 30581 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:05.156095 30582 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffc65fb71dc04d12ac503f8bf5453583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc65fb71dc04d12ac503f8bf5453583" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44941 } } }
I20260808 07:15:05.156157 30582 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583 [sys.catalog]: This master's current role is: LEADER
W20260808 07:15:05.157401 30604 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:05.157518 30604 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:05.157665 30586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:05.157836 30586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:05.158496 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:05.157429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36531 (local address 127.30.54.126:44941)
0808 07:15:05.157464 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:05.157466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:05.157506 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:15:05.157587 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:05.157587 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:05.157603 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:05.157651 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:05.158270 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:05.158271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:05.158271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:05.158275 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:05.158291 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:05.158295 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:05.158359 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:05.158417 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 07:15:05.158429 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:05.159399 30586 catalog_manager.cc:1347] Generated new cluster ID: 82bf7a62980f4f3e861848de3b9cb35c
I20260808 07:15:05.159457 30586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:05.167153 30586 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:05.167328 30586 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:05.175263 30586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffc65fb71dc04d12ac503f8bf5453583: Generated new TSK 0
I20260808 07:15:05.175436 30586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:05.176951 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 30476
2026-08-08T07:15:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:05 dist-test-slave-znh6 krb5kdc[30611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:05 dist-test-slave-znh6 krb5kdc[30611](info): set up 2 sockets
Aug 08 07:15:05 dist-test-slave-znh6 krb5kdc[30611](info): commencing operation
krb5kdc: starting...
W20260808 07:15:06.420157 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:15:06 dist-test-slave-znh6 krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:15:06.539516 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39005
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:39005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:06.620927 30631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:06.621121 30631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:06.621145 30631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:06.622205 30631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:06.622239 30631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:06.622254 30631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:06.622267 30631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:06.622282 30631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:06.623726 30631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:39005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:06.623920 30631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:06.624084 30631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:06.628924 30638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:06.628933 30641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:06.628947 30639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:06.629307 30631 server_base.cc:1034] running on GCE node
I20260808 07:15:06.629556 30631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:06.629858 30631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:06.631021 30631 hybrid_clock.cc:648] HybridClock initialized: now 1786173306631007 us; error 29 us; skew 500 ppm
Aug 08 07:15:06 dist-test-slave-znh6 krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:06.633572 30631 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:15:06.634021 30631 webserver.cc:460] Webserver started at http://127.30.54.126:44375/ using document root <none> and password file <none>
I20260808 07:15:06.634262 30631 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:06.634331 30631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:06.634485 30631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:06.635592 30631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "68e49ec252f6413a969dd10df1200856"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-znh6"
server_key: "f4bc0f988dda2576f05782260332e21d"
server_key_iv: "a95545b9db49db023d13dccc4006b435"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.635950 30631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "68e49ec252f6413a969dd10df1200856"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-znh6"
server_key: "f4bc0f988dda2576f05782260332e21d"
server_key_iv: "a95545b9db49db023d13dccc4006b435"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.637259 30631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:15:06.637959 30651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:06.638130 30631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:06.638234 30631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "68e49ec252f6413a969dd10df1200856"
format_stamp: "Formatted at 2026-08-08 07:15:06 on dist-test-slave-znh6"
server_key: "f4bc0f988dda2576f05782260332e21d"
server_key_iv: "a95545b9db49db023d13dccc4006b435"
server_key_version: "encryptionkey@0"
I20260808 07:15:06.638306 30631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:06.653052 30631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:06.655433 30631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:06.655597 30631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:06.659669 30631 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:39005
I20260808 07:15:06.659722 30723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:39005 every 8 connection(s)
I20260808 07:15:06.660085 30631 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:06.660995 30724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:06.663316 30724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856: Bootstrap starting.
I20260808 07:15:06.663910 30724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:06.664283 30724 log.cc:824] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:06.664290 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 30631
I20260808 07:15:06.664412 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:15:06.664623 30937 external_mini_cluster.cc:1428] Setting key de9625b2a7f00f5cda7da80c2918c837
I20260808 07:15:06.665084 30724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856: No bootstrap required, opened a new log
I20260808 07:15:06.667001 30724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e49ec252f6413a969dd10df1200856" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39005 } }
I20260808 07:15:06.667155 30724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:06.667220 30724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e49ec252f6413a969dd10df1200856, State: Initialized, Role: FOLLOWER
I20260808 07:15:06.667379 30724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e49ec252f6413a969dd10df1200856" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39005 } }
I20260808 07:15:06.667479 30724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:06.667524 30724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:06.667572 30724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:06.668114 30724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e49ec252f6413a969dd10df1200856" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39005 } }
I20260808 07:15:06.668232 30724 leader_election.cc:302] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68e49ec252f6413a969dd10df1200856; no voters:
I20260808 07:15:06.668413 30724 leader_election.cc:288] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:06.668449 30730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:06.668569 30730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [term 1 LEADER]: Becoming Leader. State: Replica: 68e49ec252f6413a969dd10df1200856, State: Running, Role: LEADER
I20260808 07:15:06.668689 30724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:06.668710 30730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e49ec252f6413a969dd10df1200856" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39005 } }
I20260808 07:15:06.669173 30730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68e49ec252f6413a969dd10df1200856. Latest consensus state: current_term: 1 leader_uuid: "68e49ec252f6413a969dd10df1200856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e49ec252f6413a969dd10df1200856" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39005 } } }
I20260808 07:15:06.669286 30733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68e49ec252f6413a969dd10df1200856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e49ec252f6413a969dd10df1200856" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39005 } } }
I20260808 07:15:06.669312 30730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:06.669438 30733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:06.669703 30743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:06.670287 30743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:06.671840 30743 catalog_manager.cc:1347] Generated new cluster ID: 93c9992f9a6f473dba45303b0a57eef8
I20260808 07:15:06.671898 30743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:15:06 dist-test-slave-znh6 krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:15:06.676316 30728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:06.665134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44911 (local address 127.30.54.126:39005)
0808 07:15:06.665322 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:06.665331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:06.665363 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:15:06.666285 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:06.666293 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:06.666339 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:06.666479 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:06.667170 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.668576 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.669623 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.670244 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.670348 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:06.672997 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:06.673001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:06.673002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:06.673013 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:06.674675 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.675137 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.675139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.675140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.675227 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.675445 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.675446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.675446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.675670 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:06.675860 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:06.675940 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 07:15:06.676165 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260808 07:15:06.677376 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:06.688956 30728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:06.678726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35797 (local address 127.30.54.126:39005)
0808 07:15:06.678762 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:06.678763 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:06.682994 (+ 4231us) server_negotiation.cc:407] Connection header received
0808 07:15:06.683068 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:06.683069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:06.683078 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:06.683135 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:06.683682 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.684181 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.684882 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:06.685108 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:06.685186 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:06.686354 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:06.686356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:06.686356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:06.686363 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:06.687797 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.688167 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.688168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.688168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.688229 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:06.688445 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:06.688446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:06.688446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:06.688557 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:06.688658 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:06.688768 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 07:15:06.688848 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:06.690975 30743 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:06.691464 30743 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:06.700241 30743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68e49ec252f6413a969dd10df1200856: Generated new TSK 0
I20260808 07:15:06.700459 30743 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:06.703503 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 30631
2026-08-08T07:15:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:06 dist-test-slave-znh6 krb5kdc[30766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:06 dist-test-slave-znh6 krb5kdc[30766](info): set up 2 sockets
Aug 08 07:15:06 dist-test-slave-znh6 krb5kdc[30766](info): commencing operation
krb5kdc: starting...
W20260808 07:15:07.917245 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:15:07 dist-test-slave-znh6 krb5kdc[30766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:15:08.038012 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41475
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:41977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:41475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:08.118551 30785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:08.118744 30785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:08.118767 30785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:08.119907 30785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:08.119951 30785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:08.119964 30785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:08.119976 30785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:08.119988 30785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:08.121366 30785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:41977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:41475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:41475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:08.121549 30785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:08.121701 30785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:15:08.126662 30785 server_base.cc:1034] running on GCE node
W20260808 07:15:08.126629 30791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:08.126685 30790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:08.126631 30793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:08.127075 30785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:08.127359 30785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:08.128501 30785 hybrid_clock.cc:648] HybridClock initialized: now 1786173308128494 us; error 32 us; skew 500 ppm
Aug 08 07:15:08 dist-test-slave-znh6 krb5kdc[30766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173308, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:08.134845 30785 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:15:08.135301 30785 webserver.cc:460] Webserver started at http://127.30.54.126:40033/ using document root <none> and password file <none>
I20260808 07:15:08.135524 30785 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:08.135596 30785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:08.135747 30785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:08.136796 30785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "d2ba358e94294200885dada3eb77a718"
format_stamp: "Formatted at 2026-08-08 07:15:08 on dist-test-slave-znh6"
server_key: "77a5352690aa37049853422f9e76bb1c"
server_key_iv: "8e36c3a6e727c2c86d80e2fd9a1cc2ee"
server_key_version: "encryptionkey@0"
I20260808 07:15:08.137171 30785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "d2ba358e94294200885dada3eb77a718"
format_stamp: "Formatted at 2026-08-08 07:15:08 on dist-test-slave-znh6"
server_key: "77a5352690aa37049853422f9e76bb1c"
server_key_iv: "8e36c3a6e727c2c86d80e2fd9a1cc2ee"
server_key_version: "encryptionkey@0"
I20260808 07:15:08.138526 30785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:08.139346 30802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:08.139536 30785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:08.139637 30785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "d2ba358e94294200885dada3eb77a718"
format_stamp: "Formatted at 2026-08-08 07:15:08 on dist-test-slave-znh6"
server_key: "77a5352690aa37049853422f9e76bb1c"
server_key_iv: "8e36c3a6e727c2c86d80e2fd9a1cc2ee"
server_key_version: "encryptionkey@0"
I20260808 07:15:08.139724 30785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:08.158214 30785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:08.160255 30785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:08.160429 30785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:08.164994 30785 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:41475
I20260808 07:15:08.165066 30871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:41475 every 8 connection(s)
I20260808 07:15:08.165377 30785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:08.166203 30872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:08.168514 30872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718: Bootstrap starting.
I20260808 07:15:08.169037 30872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:08.169293 30872 log.cc:824] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:08.169957 30872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718: No bootstrap required, opened a new log
I20260808 07:15:08.171605 30872 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ba358e94294200885dada3eb77a718" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41475 } }
I20260808 07:15:08.171738 30872 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:08.171761 30872 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2ba358e94294200885dada3eb77a718, State: Initialized, Role: FOLLOWER
I20260808 07:15:08.171896 30872 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ba358e94294200885dada3eb77a718" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41475 } }
I20260808 07:15:08.171975 30872 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:08.171998 30872 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:08.172025 30872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:08.172554 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 30785
I20260808 07:15:08.172701 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:15:08.172937 30937 external_mini_cluster.cc:1428] Setting key 5d8f1f0cba801d2eb2796805b45c9136
I20260808 07:15:08.173067 30872 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ba358e94294200885dada3eb77a718" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41475 } }
I20260808 07:15:08.173190 30872 leader_election.cc:302] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2ba358e94294200885dada3eb77a718; no voters:
I20260808 07:15:08.173405 30872 leader_election.cc:288] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:08.173431 30878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:08.173614 30878 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [term 1 LEADER]: Becoming Leader. State: Replica: d2ba358e94294200885dada3eb77a718, State: Running, Role: LEADER
I20260808 07:15:08.173667 30872 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:08.173704 30878 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ba358e94294200885dada3eb77a718" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41475 } }
I20260808 07:15:08.174026 30880 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2ba358e94294200885dada3eb77a718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ba358e94294200885dada3eb77a718" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41475 } } }
I20260808 07:15:08.174124 30880 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:08.174288 30882 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2ba358e94294200885dada3eb77a718. Latest consensus state: current_term: 1 leader_uuid: "d2ba358e94294200885dada3eb77a718" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ba358e94294200885dada3eb77a718" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 41475 } } }
I20260808 07:15:08.174350 30882 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:08.174485 30889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:08.175037 30889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:08.178761 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.173446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38661 (local address 127.30.54.126:41475)
0808 07:15:08.173616 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.173626 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.173753 (+ 127us) server_negotiation.cc:407] Connection header received
0808 07:15:08.173994 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.174002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.174057 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.174178 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:08.174613 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.176128 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.176961 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.177360 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.177482 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.178159 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:08.178162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:08.178163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:08.178327 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:08.178357 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:08.178367 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:08.178450 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.178527 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.178587 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260808 07:15:08.179695 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:08.179885 30889 catalog_manager.cc:1347] Generated new cluster ID: 2b6d5b1c175848d5adbe4de87cec03cf
I20260808 07:15:08.179941 30889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:08.185645 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:08.181109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44739 (local address 127.30.54.126:41475)
0808 07:15:08.181157 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:08.181159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:08.183000 (+ 1841us) server_negotiation.cc:407] Connection header received
0808 07:15:08.183070 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:08.183070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:08.183080 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:08.183135 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:08.183273 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.183744 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.184434 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:08.184644 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:08.184716 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:08.185387 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:08.185388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:08.185389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:08.185394 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:08.185413 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:08.185417 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:08.185470 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:08.185529 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 07:15:08.185552 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:15:08.190606 30889 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:08.191157 30889 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:08.193858 30889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2ba358e94294200885dada3eb77a718: Generated new TSK 0
I20260808 07:15:08.194018 30889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:08.199070 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 30785
2026-08-08T07:15:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:08 dist-test-slave-znh6 krb5kdc[30915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:08 dist-test-slave-znh6 krb5kdc[30915](info): set up 2 sockets
Aug 08 07:15:08 dist-test-slave-znh6 krb5kdc[30915](info): commencing operation
krb5kdc: starting...
W20260808 07:15:09.411545 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:15:09 dist-test-slave-znh6 krb5kdc[30915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:15:09.534353 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:46781
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:36425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:46781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:09.614053 30934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:09.614234 30934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:09.614255 30934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:09.615319 30934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:09.615361 30934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:09.615375 30934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:09.615387 30934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:09.615399 30934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:09.616765 30934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:36425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:46781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:46781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:09.616937 30934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:09.617097 30934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:09.621979 30948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:09.621987 30947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:09.622004 30934 server_base.cc:1034] running on GCE node
W20260808 07:15:09.621980 30950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:09.622437 30934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:09.622705 30934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:09.623852 30934 hybrid_clock.cc:648] HybridClock initialized: now 1786173309623848 us; error 35 us; skew 500 ppm
Aug 08 07:15:09 dist-test-slave-znh6 krb5kdc[30915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173309, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:09.626300 30934 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:15:09.626767 30934 webserver.cc:460] Webserver started at http://127.30.54.126:40863/ using document root <none> and password file <none>
I20260808 07:15:09.627030 30934 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:09.627101 30934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:09.627259 30934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:09.628265 30934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "52ec3e35febf428190526c269844c1f5"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-znh6"
server_key: "68c5629a4ee7eeb2288a978b1c7c254f"
server_key_iv: "3829fa647113517b64be33ef875092d3"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.628626 30934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "52ec3e35febf428190526c269844c1f5"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-znh6"
server_key: "68c5629a4ee7eeb2288a978b1c7c254f"
server_key_iv: "3829fa647113517b64be33ef875092d3"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.629945 30934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:09.630723 30960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:09.630932 30934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:15:09.631034 30934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "52ec3e35febf428190526c269844c1f5"
format_stamp: "Formatted at 2026-08-08 07:15:09 on dist-test-slave-znh6"
server_key: "68c5629a4ee7eeb2288a978b1c7c254f"
server_key_iv: "3829fa647113517b64be33ef875092d3"
server_key_version: "encryptionkey@0"
I20260808 07:15:09.631120 30934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:09.634290 30934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:09.636582 30934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:09.636741 30934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:09.640769 30934 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:46781
I20260808 07:15:09.640800 31033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:46781 every 8 connection(s)
I20260808 07:15:09.641111 30934 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:09.642050 31035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:09.644204 31035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5: Bootstrap starting.
I20260808 07:15:09.644795 31035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:09.645113 31035 log.cc:824] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:09.645931 31035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5: No bootstrap required, opened a new log
I20260808 07:15:09.647624 31035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ec3e35febf428190526c269844c1f5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46781 } }
I20260808 07:15:09.647753 31035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:15:09.647782 31035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ec3e35febf428190526c269844c1f5, State: Initialized, Role: FOLLOWER
I20260808 07:15:09.647960 31035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ec3e35febf428190526c269844c1f5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46781 } }
I20260808 07:15:09.648042 31035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:15:09.648064 31035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:15:09.648131 31035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:09.648980 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 30934
I20260808 07:15:09.649063 31035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ec3e35febf428190526c269844c1f5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46781 } }
I20260808 07:15:09.649117 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:15:09.649170 31035 leader_election.cc:302] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52ec3e35febf428190526c269844c1f5; no voters:
I20260808 07:15:09.649334 31035 leader_election.cc:288] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:15:09.649343 30937 external_mini_cluster.cc:1428] Setting key 42ef48b064cdc49802a0bda136560f65
I20260808 07:15:09.649392 31039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:09.649580 31039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [term 1 LEADER]: Becoming Leader. State: Replica: 52ec3e35febf428190526c269844c1f5, State: Running, Role: LEADER
I20260808 07:15:09.649753 31039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ec3e35febf428190526c269844c1f5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46781 } }
I20260808 07:15:09.649616 31035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:09.650206 31040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52ec3e35febf428190526c269844c1f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ec3e35febf428190526c269844c1f5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46781 } } }
I20260808 07:15:09.650300 31040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:09.650436 31040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52ec3e35febf428190526c269844c1f5. Latest consensus state: current_term: 1 leader_uuid: "52ec3e35febf428190526c269844c1f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52ec3e35febf428190526c269844c1f5" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 46781 } } }
I20260808 07:15:09.650491 31040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:09.650933 31047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:09.651486 31047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:09.652465 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:09.649800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56873 (local address 127.30.54.126:46781)
0808 07:15:09.649985 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:09.649994 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:09.650026 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:15:09.650202 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:09.650210 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:09.650240 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:09.650673 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:09.651843 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:09.651847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:09.651847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:09.651851 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:09.651892 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:09.651906 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:09.651987 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:09.652248 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 07:15:09.652287 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260808 07:15:09.653026 31047 catalog_manager.cc:1347] Generated new cluster ID: e40a914bdeb24d92ad357f2fa30a9d2e
I20260808 07:15:09.653075 31047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:09.653455 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:09.656036 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:09.655013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56423 (local address 127.30.54.126:46781)
0808 07:15:09.655046 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:09.655048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:09.655173 (+ 125us) server_negotiation.cc:407] Connection header received
0808 07:15:09.655200 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:09.655201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:09.655207 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:09.655277 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:09.655825 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:09.655827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:09.655827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:15:09.655831 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:09.655845 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:15:09.655849 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:09.655913 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:09.655967 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 07:15:09.655979 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:15:09.657641 31047 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:09.657788 31047 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:09.664731 31047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52ec3e35febf428190526c269844c1f5: Generated new TSK 0
I20260808 07:15:09.664914 31047 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:15:09.670746 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 30934
2026-08-08T07:15:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1471 ms)
[----------] 18 tests from AuthTokenIssuingTest (26999 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:09 dist-test-slave-znh6 krb5kdc[31072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:09 dist-test-slave-znh6 krb5kdc[31072](info): set up 2 sockets
Aug 08 07:15:09 dist-test-slave-znh6 krb5kdc[31072](info): commencing operation
krb5kdc: starting...
W20260808 07:15:10.896767 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:15:10 dist-test-slave-znh6 krb5kdc[31072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:15:11.013717 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:44553
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:44553,127.30.54.125:46533,127.30.54.124:36521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:11.093379 31097 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:11.093573 31097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:11.093592 31097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:11.093611 31097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:11.094637 31097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:11.094671 31097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:11.094686 31097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:11.094698 31097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:11.094710 31097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:11.096186 31097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:44553,127.30.54.125:46533,127.30.54.124:36521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:44553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:11.096374 31097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:11.096534 31097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:11.101534 31104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:11.101539 31107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:11.101554 31097 server_base.cc:1034] running on GCE node
W20260808 07:15:11.101557 31109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:11.102026 31097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:11.102314 31097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:11.103477 31097 hybrid_clock.cc:648] HybridClock initialized: now 1786173311103473 us; error 37 us; skew 500 ppm
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:11.105816 31097 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:15:11.106232 31097 webserver.cc:460] Webserver started at http://127.30.54.126:43869/ using document root <none> and password file <none>
I20260808 07:15:11.106463 31097 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:11.106535 31097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:11.106686 31097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:11.107743 31097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "40813514af9141a5959ed23d8e12fcfa"
format_stamp: "Formatted at 2026-08-08 07:15:11 on dist-test-slave-znh6"
server_key: "41e5212431f3cf141a9004b82d5ae887"
server_key_iv: "3c63e6ce3c6b9e89ab10644f8f2aafbe"
server_key_version: "encryptionkey@0"
I20260808 07:15:11.108103 31097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "40813514af9141a5959ed23d8e12fcfa"
format_stamp: "Formatted at 2026-08-08 07:15:11 on dist-test-slave-znh6"
server_key: "41e5212431f3cf141a9004b82d5ae887"
server_key_iv: "3c63e6ce3c6b9e89ab10644f8f2aafbe"
server_key_version: "encryptionkey@0"
I20260808 07:15:11.109365 31097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:15:11.110069 31117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:11.110244 31097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:11.110342 31097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "40813514af9141a5959ed23d8e12fcfa"
format_stamp: "Formatted at 2026-08-08 07:15:11 on dist-test-slave-znh6"
server_key: "41e5212431f3cf141a9004b82d5ae887"
server_key_iv: "3c63e6ce3c6b9e89ab10644f8f2aafbe"
server_key_version: "encryptionkey@0"
I20260808 07:15:11.110428 31097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:11.117645 31097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:11.119532 31097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:11.119684 31097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:11.123518 31097 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:44553
I20260808 07:15:11.123549 31195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:44553 every 8 connection(s)
I20260808 07:15:11.123870 31097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:11.124888 31197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:11.126253 31197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:11.128126 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31097
I20260808 07:15:11.128243 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:15:11.128434 30937 external_mini_cluster.cc:1428] Setting key 6bcf0b0e1bd9e53e30ba2e920770c2ad
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:15:11.136684 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.126877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53820 (local address 127.30.54.126:44553)
0808 07:15:11.127035 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.127043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.127154 (+ 111us) server_negotiation.cc:407] Connection header received
0808 07:15:11.127708 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.127710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.127734 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.127829 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.128528 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.129375 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.130563 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.131124 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.131258 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.133580 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.133588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.133588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.133594 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.135158 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.135614 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.135616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.135617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.135688 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.135964 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.135964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.135965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.136144 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.136203 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.136454 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.136544 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260808 07:15:11.136806 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.126793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:44553 (local address 127.0.0.1:53820)
0808 07:15:11.127022 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.127064 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.127551 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.127834 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.127836 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.128168 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:11.128403 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.128410 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.129484 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.129485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.130478 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.130481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.131237 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.131238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.131294 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.131790 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:11.131798 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:11.133484 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:11.135242 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.135250 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.135253 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.135525 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.135768 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.135769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.135769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.135884 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.136224 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:11.136232 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:11.136310 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.136411 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.136650 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":74,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.30.54.125@KRBTEST.COM; defaulting to no policy
I20260808 07:15:11.137542 31197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:11.137862 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.137750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:46533 (local address 127.0.0.1:41078)
0808 07:15:11.137796 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.137834 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:15:11.138021 31124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
I20260808 07:15:11.138430 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.138351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:46533 (local address 127.0.0.1:41094)
0808 07:15:11.138392 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.138415 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:15:11.138603 31197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.125:46533: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab.
Entry for principal kudu/127.30.54.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab.
I20260808 07:15:11.153733 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.125:46533
--webserver_interface=127.30.54.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab
--principal=kudu/127.30.54.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:44553,127.30.54.125:46533,127.30.54.124:36521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:15:11.187821 31197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } attempt: 1
I20260808 07:15:11.188279 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.188143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:46533 (local address 127.0.0.1:41096)
0808 07:15:11.188207 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.188247 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:15:11.188628 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.188530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:46533 (local address 127.0.0.1:41098)
0808 07:15:11.188576 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.188603 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:15:11.188715 31197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.125:46533: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
W20260808 07:15:11.233992 31206 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:11.234185 31206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:11.234203 31206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:11.234222 31206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:11.235380 31206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:11.235419 31206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:11.235433 31206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:11.235445 31206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:11.235458 31206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:11.236934 31206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:44553,127.30.54.125:46533,127.30.54.124:36521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab
--principal=kudu/127.30.54.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.125:46533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.54.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:11.237169 31206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:11.237392 31206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:11.242388 31218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:11.242374 31215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:11.242372 31211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:11.242408 31206 server_base.cc:1034] running on GCE node
I20260808 07:15:11.242930 31206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:11.243206 31206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:11.244401 31206 hybrid_clock.cc:648] HybridClock initialized: now 1786173311244379 us; error 32 us; skew 500 ppm
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:11.246804 31206 init.cc:377] Logged in from keytab as kudu/127.30.54.125@KRBTEST.COM (short username kudu)
I20260808 07:15:11.247273 31206 webserver.cc:460] Webserver started at http://127.30.54.125:32775/ using document root <none> and password file <none>
I20260808 07:15:11.247493 31206 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:11.247562 31206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:11.247720 31206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:11.248832 31206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/data/instance:
uuid: "ac96d1f70b50490593c814a7ec878df4"
format_stamp: "Formatted at 2026-08-08 07:15:11 on dist-test-slave-znh6"
server_key: "7b9132cc07c97e075eeff2dc2d69f201"
server_key_iv: "7a53b383de5037edf5a3bf239392b7e2"
server_key_version: "encryptionkey@0"
I20260808 07:15:11.249192 31206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/wal/instance:
uuid: "ac96d1f70b50490593c814a7ec878df4"
format_stamp: "Formatted at 2026-08-08 07:15:11 on dist-test-slave-znh6"
server_key: "7b9132cc07c97e075eeff2dc2d69f201"
server_key_iv: "7a53b383de5037edf5a3bf239392b7e2"
server_key_version: "encryptionkey@0"
I20260808 07:15:11.250761 31206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:15:11.251528 31225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:11.251711 31206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:15:11.251807 31206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/wal
uuid: "ac96d1f70b50490593c814a7ec878df4"
format_stamp: "Formatted at 2026-08-08 07:15:11 on dist-test-slave-znh6"
server_key: "7b9132cc07c97e075eeff2dc2d69f201"
server_key_iv: "7a53b383de5037edf5a3bf239392b7e2"
server_key_version: "encryptionkey@0"
I20260808 07:15:11.251902 31206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:11.256866 31197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } attempt: 2
I20260808 07:15:11.257265 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.257145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:46533 (local address 127.0.0.1:41102)
0808 07:15:11.257204 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.257238 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:15:11.257596 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.257510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:46533 (local address 127.0.0.1:41108)
0808 07:15:11.257552 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.257577 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:15:11.257745 31197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.125:46533: Network error: Client connection negotiation failed: client connection to 127.30.54.125:46533: connect: Connection refused (error 111)
I20260808 07:15:11.277912 31206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:11.279721 31206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:11.279887 31206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:11.283900 31206 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.125:46533
I20260808 07:15:11.283955 31291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.125:46533 every 8 connection(s)
I20260808 07:15:11.284281 31206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
I20260808 07:15:11.285270 31292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:11.286684 31292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:11.288347 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31206
I20260808 07:15:11.288493 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-1/wal/instance
I20260808 07:15:11.288686 30937 external_mini_cluster.cc:1428] Setting key 51bb18e62de3542d74c5d8f60743d82b
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:15:11.296412 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.287360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53824 (local address 127.30.54.126:44553)
0808 07:15:11.287430 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.287433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.287657 (+ 224us) server_negotiation.cc:407] Connection header received
0808 07:15:11.288074 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.288075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.288087 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.288139 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.289004 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.289476 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.290871 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.291202 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.291276 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.293356 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.293358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.293358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.293366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.295023 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.295439 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.295440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.295441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.295502 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.295761 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.295763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.295763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.295882 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.295963 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.296225 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.296311 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:15:11.296578 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.287345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:44553 (local address 127.0.0.1:53824)
0808 07:15:11.287537 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.287570 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.287992 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.288248 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.288253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.288620 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:11.288913 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.288920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.289569 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.289570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.290755 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.290758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.291307 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.291308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.291419 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.291885 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:11.291901 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:11.293262 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:11.295100 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.295106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.295107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.295310 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.295583 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.295583 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.295584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.295692 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.295954 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:11.295961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:11.296096 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.296204 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.296350 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.30.54.124@KRBTEST.COM; defaulting to no policy
I20260808 07:15:11.297164 31292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.124@KRBTEST.COM" created.
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
I20260808 07:15:11.305119 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.297490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.30.54.125:46533)
0808 07:15:11.297677 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.297682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.297732 (+ 50us) server_negotiation.cc:407] Connection header received
0808 07:15:11.297756 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.297758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.297772 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.297860 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.298135 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.298767 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.299567 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.299852 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.299932 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.302108 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.302116 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.302117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.302125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.303719 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.304177 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.304180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.304181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.304247 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.304477 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.304478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.304478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.304647 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.304748 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.304905 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.305036 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260808 07:15:11.305169 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.297426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:46533 (local address 127.0.0.1:41112)
0808 07:15:11.297468 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.297476 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.297520 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.297893 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.297894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.298008 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:11.298060 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.298061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.298898 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.298899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.299487 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.299489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.299936 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.299937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.300022 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.300423 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:11.300429 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:11.301992 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:11.303818 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.303819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.303820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.304068 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.304331 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.304332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.304333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.304406 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.304740 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:11.304741 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:11.304798 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.304882 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.304990 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 07:15:11.306070 31292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:11.306468 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.306300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45828)
0808 07:15:11.306357 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.306413 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:15:11.306614 31227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
I20260808 07:15:11.307116 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.307002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45840)
0808 07:15:11.307057 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.307089 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:15:11.307248 31292 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:36521: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab.
Entry for principal kudu/127.30.54.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab.
I20260808 07:15:11.317149 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.124:36521
--webserver_interface=127.30.54.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab
--principal=kudu/127.30.54.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:44553,127.30.54.125:46533,127.30.54.124:36521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:15:11.348927 31197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } attempt: 3
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
I20260808 07:15:11.356256 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.349274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:46533 (local address 127.0.0.1:41114)
0808 07:15:11.349330 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.349342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.349420 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.349579 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.349580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.349753 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:11.349820 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.349821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.350533 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.350534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.351161 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.351164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.351578 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.351579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.351664 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.352163 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:11.352167 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:11.353349 (+ 1182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:11.355098 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.355099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.355100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.355334 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.355596 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.355596 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.355597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.355661 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.355919 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:11.355921 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:11.355966 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.356054 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.356155 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:15:11.356254 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.349351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.30.54.125:46533)
0808 07:15:11.349425 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.349426 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.349437 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:15:11.349487 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.349488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.349500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.349573 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.349917 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.350437 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.351256 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.351485 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.351577 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.353459 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.353460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.353461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.353468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.355003 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.355444 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.355445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.355445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.355521 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.355730 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.355731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.355731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.355842 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.355919 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.356080 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.356156 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:15:11.356434 31292 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } attempt: 1
I20260808 07:15:11.356637 31197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:11.356909 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.356802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45842)
0808 07:15:11.356845 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.356879 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 07:15:11.356951 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.356857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45854)
0808 07:15:11.356903 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.356925 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 07:15:11.357302 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.357180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45860)
0808 07:15:11.357237 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.357273 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:15:11.357445 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.357181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45862)
0808 07:15:11.357395 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.357421 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200}
W20260808 07:15:11.357435 31197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:36521: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
W20260808 07:15:11.357533 31292 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:36521: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
I20260808 07:15:11.388607 31197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } attempt: 1
I20260808 07:15:11.389118 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.388941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45876)
0808 07:15:11.389022 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.389072 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 07:15:11.389542 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.389436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45890)
0808 07:15:11.389483 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.389515 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260808 07:15:11.389724 31197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:36521: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
W20260808 07:15:11.400547 31301 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:11.400833 31301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:11.400864 31301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:11.400915 31301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:11.402915 31301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:11.402969 31301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:11.402992 31301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:11.403009 31301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:11.403025 31301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:11.405691 31301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:44553,127.30.54.125:46533,127.30.54.124:36521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab
--principal=kudu/127.30.54.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.124:36521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.54.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:11.405962 31301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:11.406206 31301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:11.411818 31307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:11.411872 31306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:11.411873 31311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:11.412147 31301 server_base.cc:1034] running on GCE node
I20260808 07:15:11.412395 31301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:11.412705 31301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:11.413883 31301 hybrid_clock.cc:648] HybridClock initialized: now 1786173311413870 us; error 27 us; skew 500 ppm
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:11.419062 31301 init.cc:377] Logged in from keytab as kudu/127.30.54.124@KRBTEST.COM (short username kudu)
I20260808 07:15:11.419593 31301 webserver.cc:460] Webserver started at http://127.30.54.124:44409/ using document root <none> and password file <none>
I20260808 07:15:11.419849 31301 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:11.419916 31301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:11.420061 31301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:11.421386 31301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/data/instance:
uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8"
format_stamp: "Formatted at 2026-08-08 07:15:11 on dist-test-slave-znh6"
server_key: "7a9115f8c26b6be5e9b23927c6a752a9"
server_key_iv: "621dccf667bbb624a85738e2931ece8f"
server_key_version: "encryptionkey@0"
I20260808 07:15:11.421838 31301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/wal/instance:
uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8"
format_stamp: "Formatted at 2026-08-08 07:15:11 on dist-test-slave-znh6"
server_key: "7a9115f8c26b6be5e9b23927c6a752a9"
server_key_iv: "621dccf667bbb624a85738e2931ece8f"
server_key_version: "encryptionkey@0"
I20260808 07:15:11.423503 31301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 07:15:11.424589 31324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:11.424801 31301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:11.424901 31301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/wal
uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8"
format_stamp: "Formatted at 2026-08-08 07:15:11 on dist-test-slave-znh6"
server_key: "7a9115f8c26b6be5e9b23927c6a752a9"
server_key_iv: "621dccf667bbb624a85738e2931ece8f"
server_key_version: "encryptionkey@0"
I20260808 07:15:11.424973 31301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:11.426846 31292 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } attempt: 2
I20260808 07:15:11.428213 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.428055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45896)
0808 07:15:11.428132 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.428175 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 07:15:11.428558 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.428460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45910)
0808 07:15:11.428510 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.428538 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:15:11.428659 31292 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:36521: Network error: Client connection negotiation failed: client connection to 127.30.54.124:36521: connect: Connection refused (error 111)
I20260808 07:15:11.447172 31301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:11.449100 31301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:11.449258 31301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:11.453753 31301 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.124:36521
I20260808 07:15:11.453761 31394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.124:36521 every 8 connection(s)
I20260808 07:15:11.454169 31301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
I20260808 07:15:11.455265 31395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:11.456851 31395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:11.462122 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31301
I20260808 07:15:11.462212 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786173009519897-30937-0/minicluster-data/master-2/wal/instance
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:15:11.462388 30937 external_mini_cluster.cc:1428] Setting key 50bb3fd2e84141cfc398130dec8d7883
I20260808 07:15:11.464895 31197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } attempt: 2
I20260808 07:15:11.465942 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.457559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53834 (local address 127.30.54.126:44553)
0808 07:15:11.457606 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.457609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.457780 (+ 171us) server_negotiation.cc:407] Connection header received
0808 07:15:11.458225 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.458226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.458236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.458296 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.458946 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.459415 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.460367 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.460549 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.460606 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.462900 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.462902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.462902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.462910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.464395 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.464857 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.464859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.464859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.464928 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.465234 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.465235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.465236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.465380 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.465458 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.465758 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.465845 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:15:11.466171 31397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.457481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:44553 (local address 127.0.0.1:53834)
0808 07:15:11.457650 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.457690 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.458145 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.458372 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.458375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.458615 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:11.458809 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.458869 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.459493 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.459493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.460286 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.460289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.460635 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.460636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.460739 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.461175 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:11.461183 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:11.462752 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:11.464477 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.464482 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.464484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.464777 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.465011 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.465012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.465013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.465153 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.465461 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:11.465467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:11.465618 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.465730 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.465919 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":25,"thread_start_us":96,"threads_started":1}
I20260808 07:15:11.466741 31395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
I20260808 07:15:11.473151 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.465132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45920)
0808 07:15:11.465183 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.465194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.465228 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.465476 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.465476 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.465602 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:11.465655 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.465656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.466572 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.466573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.467358 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.467361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.467770 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.467771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.467846 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.468234 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:11.468239 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:11.469623 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:11.471945 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.471946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.471946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.472154 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.472386 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.472387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.472387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.472449 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.472824 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:11.472825 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:11.472870 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.472948 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.473050 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:15:11.473299 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.462805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.30.54.124:36521)
0808 07:15:11.463024 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.463029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.463054 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:15:11.463105 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.463108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.463120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.463211 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.463873 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.464606 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.465675 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.466116 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.466185 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.469839 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.469841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.469841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.469848 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.471753 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.472238 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.472239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.472239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.472335 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.472580 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.472581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.472581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.472721 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.472863 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.473072 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.473148 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":77,"threads_started":1}
I20260808 07:15:11.473196 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.465190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.30.54.124:36521)
0808 07:15:11.465347 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.465349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.465355 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:15:11.465373 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.465374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.465383 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.465728 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.465752 (+ 24us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.466485 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.467444 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.467692 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.467769 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.469719 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.469728 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.469728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.469735 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.471866 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.472230 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.472232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.472233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.472308 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.472522 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.472523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.472523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.472718 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.472826 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.473000 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.473084 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260808 07:15:11.474776 31197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa: Bootstrap starting.
I20260808 07:15:11.475615 31197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:11.475924 31197 log.cc:824] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:11.476680 31197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa: No bootstrap required, opened a new log
I20260808 07:15:11.478019 31397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.467019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:46533 (local address 127.0.0.1:41128)
0808 07:15:11.467061 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.467071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.467105 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.467255 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.467255 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.467362 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:11.467436 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.467438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.468359 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.468360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.469038 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.469040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.469523 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.469524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.469608 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.470034 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:11.470038 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:11.472053 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:11.476837 (+ 4784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.476838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.476839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.477073 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.477292 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.477292 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.477293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.477367 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.477661 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:11.477662 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:11.477712 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.477799 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.477892 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 07:15:11.478020 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.467086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.30.54.125:46533)
0808 07:15:11.467131 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.467134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.467142 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:11.467166 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.467167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.467176 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.467260 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.467505 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.468266 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.469122 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.469442 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.469515 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.472143 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.472144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.472145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.472152 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.476744 (+ 4592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.477156 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.477157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.477157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.477223 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.477440 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.477441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.477441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.477577 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.477649 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.477846 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.477931 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:15:11.478652 31395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:11.478789 31197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } }
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:15:11.478941 31197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40813514af9141a5959ed23d8e12fcfa, State: Initialized, Role: FOLLOWER
I20260808 07:15:11.479020 31197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } }
I20260808 07:15:11.479372 31197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:11.479638 31403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.479810 31403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:15:11.481247 31418 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:11.481688 31418 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:11.482707 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.474127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.30.54.126:44553)
0808 07:15:11.474164 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.474166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.474209 (+ 43us) server_negotiation.cc:407] Connection header received
0808 07:15:11.474282 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.474282 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.474290 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.474341 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.474809 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.475297 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.475967 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.476198 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.476284 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.479454 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.479455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.479455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.479463 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.481083 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.481881 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.481882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.481883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.481943 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.482166 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.482167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.482167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.482301 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.482422 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.482551 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.482630 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
I20260808 07:15:11.482960 31097 master_runner.cc:186] Error getting master registration for 127.30.54.125:46533: OK, instance_id { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" instance_seqno: 1786173311282678 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:15:11.486238 31097 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:15:11.487466 31397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.478934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45934)
0808 07:15:11.478976 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.478985 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.479020 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.479246 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.479247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.479391 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:11.479475 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.479478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.480669 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.480670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.481403 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.481406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.482587 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.482588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.482637 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.483043 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:11.483048 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:11.484453 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:11.486387 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.486388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.486388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.486604 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.486804 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.486804 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.486805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.486881 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.487192 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:11.487193 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:11.487232 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.487275 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.487370 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 07:15:11.487879 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.479112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.30.54.124:36521)
0808 07:15:11.479144 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.479146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.479153 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:11.479171 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.479171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.479179 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.479236 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.479575 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.480575 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.482209 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.482503 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.482789 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.484542 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.484543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.484544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.484549 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.486170 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.486675 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.486676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.486676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.486727 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.486957 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.486959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.486959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.487107 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.487635 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.487678 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.487766 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
I20260808 07:15:11.489181 31395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8: Bootstrap starting.
I20260808 07:15:11.490234 31395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:11.490545 31395 log.cc:824] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:11.491416 31395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8: No bootstrap required, opened a new log
I20260808 07:15:11.492231 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.484107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.30.54.125:46533)
0808 07:15:11.484144 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.484146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.484219 (+ 73us) server_negotiation.cc:407] Connection header received
0808 07:15:11.484364 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.484365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.484373 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.484402 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.485048 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.485729 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.486632 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.486957 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.487036 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.489423 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.489424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.489424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.489431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.491037 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.491383 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.491384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.491384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.491440 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.491682 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.491683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.491684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.491796 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.491952 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.491990 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.492062 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:11.493399 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:11.493535 31395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } }
I20260808 07:15:11.493686 31395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b3ec5e6f7541298cbc6ca7e6d1e8b8, State: Initialized, Role: FOLLOWER
I20260808 07:15:11.493769 31395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } }
I20260808 07:15:11.494069 31395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:11.494045 31421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.494238 31421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:11.494415 31154 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "40813514af9141a5959ed23d8e12fcfa"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53846
I20260808 07:15:11.494558 31154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:15:11.494620 31154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:11.495668 31154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } }
I20260808 07:15:11.495924 31154 leader_election.cc:288] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [CANDIDATE]: Term 1 election: Requested vote from peers ac96d1f70b50490593c814a7ec878df4 (127.30.54.125:46533), 83b3ec5e6f7541298cbc6ca7e6d1e8b8 (127.30.54.124:36521)
I20260808 07:15:11.496289 31360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "40813514af9141a5959ed23d8e12fcfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8"
I20260808 07:15:11.497555 31360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:11.498450 31360 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40813514af9141a5959ed23d8e12fcfa in term 1.
W20260808 07:15:11.496354 31438 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:11.498741 31438 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:11.498768 31121 leader_election.cc:302] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40813514af9141a5959ed23d8e12fcfa, 83b3ec5e6f7541298cbc6ca7e6d1e8b8; no voters:
I20260808 07:15:11.498896 31403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:11.499082 31403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [term 1 LEADER]: Becoming Leader. State: Replica: 40813514af9141a5959ed23d8e12fcfa, State: Running, Role: LEADER
I20260808 07:15:11.499171 31403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } }
I20260808 07:15:11.499612 31448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40813514af9141a5959ed23d8e12fcfa. Latest consensus state: current_term: 1 leader_uuid: "40813514af9141a5959ed23d8e12fcfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.499718 31448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:11.500542 31301 master_runner.cc:186] Error getting master registration for 127.30.54.125:46533: OK, instance_id { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" instance_seqno: 1786173311282678 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:15:11.502717 31301 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:15:11.510931 31450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:11.511160 31450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:11.511626 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.503278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53850 (local address 127.30.54.126:44553)
0808 07:15:11.503315 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.503316 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.503724 (+ 408us) server_negotiation.cc:407] Connection header received
0808 07:15:11.503839 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.503840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.503851 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.503883 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.504591 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.505322 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.506368 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.506672 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.506712 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.508394 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.508396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.508397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.508404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.510132 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.510552 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.510553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.510554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.510613 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.510882 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.510883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.510883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.511032 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.511104 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.511449 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.511519 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:11.513350 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.503070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.30.54.124:36521)
0808 07:15:11.503106 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.503108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.503247 (+ 139us) server_negotiation.cc:407] Connection header received
0808 07:15:11.503395 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.503396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.503406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.503457 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.504665 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.505360 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.506904 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.507208 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.507250 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.509920 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.509921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.509922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.509929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.511846 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.512313 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.512314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.512314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.512367 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.512607 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.512608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.512608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.512761 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.512878 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.513160 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.513247 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:11.518108 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.503508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.30.54.125:46533)
0808 07:15:11.503551 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.503553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.510988 (+ 7435us) server_negotiation.cc:407] Connection header received
0808 07:15:11.511091 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.511092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.511101 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.511130 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.511789 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.512525 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.513612 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.513921 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.513961 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.515490 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.515491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.515491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.515498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.516832 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.517230 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.517231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.517231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.517306 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.517507 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.517508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.517508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.517639 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.517746 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.517939 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.518012 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:15:11.519781 31292 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } attempt: 3
I20260808 07:15:11.519840 31360 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [term 1 FOLLOWER]: Refusing update from remote peer 40813514af9141a5959ed23d8e12fcfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:11.520179 31403 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:11.521425 31421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40813514af9141a5959ed23d8e12fcfa. Latest consensus state: current_term: 1 leader_uuid: "40813514af9141a5959ed23d8e12fcfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.521507 31421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 07:15:11 dist-test-slave-znh6 krb5kdc[31072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173311, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
I20260808 07:15:11.527639 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.519994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:36521 (local address 127.0.0.1:45948)
0808 07:15:11.520032 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:11.520055 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:11.520095 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:11.520217 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:11.520218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:11.520386 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:11.520424 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.520426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.521179 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.521180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.521967 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:11.521970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.522389 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.522390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:11.522450 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.522881 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:11.522886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:11.524709 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:11.526553 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.526554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.526555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.526723 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.526974 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:11.526975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:11.526975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:11.527038 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:11.527326 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:11.527327 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:11.527369 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 07:15:11.527434 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:15:11.527527 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:15:11.528047 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:11.520077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.30.54.124:36521)
0808 07:15:11.520109 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.520111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.520118 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:11.520138 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.520138 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.520146 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.520202 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.520503 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.521101 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.522050 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:11.522324 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:11.522389 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:11.524801 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:11.524803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:11.524803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:11.524809 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:11.526472 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.526835 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.526836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.526837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.526896 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:11.527106 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:11.527107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:11.527107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:11.527237 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:11.527299 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:11.527876 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 07:15:11.527956 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:15:11.529632 31292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4: Bootstrap starting.
I20260808 07:15:11.530279 31292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:11.530579 31292 log.cc:824] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:11.531517 31292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4: No bootstrap required, opened a new log
I20260808 07:15:11.533514 31292 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } }
I20260808 07:15:11.533648 31292 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac96d1f70b50490593c814a7ec878df4, State: Initialized, Role: FOLLOWER
I20260808 07:15:11.533730 31292 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } }
I20260808 07:15:11.534060 31292 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:11.534132 31255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:11.535130 31255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [term 1 FOLLOWER]: Refusing update from remote peer 40813514af9141a5959ed23d8e12fcfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:11.534022 31463 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.535444 31463 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:11.535717 31403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "40813514af9141a5959ed23d8e12fcfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.535820 31403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:11.536180 31450 catalog_manager.cc:1347] Generated new cluster ID: ecb665021ae246d68f4a9fd2544ae9c6
I20260808 07:15:11.536235 31450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:11.536226 31448 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:11.536679 31421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "40813514af9141a5959ed23d8e12fcfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.536759 31421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:11.538004 31463 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40813514af9141a5959ed23d8e12fcfa. Latest consensus state: current_term: 1 leader_uuid: "40813514af9141a5959ed23d8e12fcfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.538115 31463 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:11.538161 31463 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "40813514af9141a5959ed23d8e12fcfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.538141 31256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "40813514af9141a5959ed23d8e12fcfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ac96d1f70b50490593c814a7ec878df4"
I20260808 07:15:11.538201 31463 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:11.538329 31457 mvcc.cc:205] Tried to move back new op lower bound from 7316165883982290944 to 7316165883901317120. Current Snapshot: MvccSnapshot[applied={T|T < 7316165883982290944 or (T in {7316165883982290944})}]
I20260808 07:15:11.539469 31256 raft_consensus.cc:2396] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 40813514af9141a5959ed23d8e12fcfa for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260808 07:15:11.540231 31484 catalog_manager.cc:1259] Loaded cluster ID: ecb665021ae246d68f4a9fd2544ae9c6
I20260808 07:15:11.546386 31484 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4: loading cluster ID for follower catalog manager: success
W20260808 07:15:11.546536 31484 catalog_manager.cc:1576] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 07:15:11.546581 31484 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:11.570837 31448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "40813514af9141a5959ed23d8e12fcfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40813514af9141a5959ed23d8e12fcfa" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 44553 } } peers { permanent_uuid: "ac96d1f70b50490593c814a7ec878df4" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 46533 } } peers { permanent_uuid: "83b3ec5e6f7541298cbc6ca7e6d1e8b8" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 36521 } } }
I20260808 07:15:11.570968 31448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:11.572444 31450 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:11.573179 31450 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:11.598902 31450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40813514af9141a5959ed23d8e12fcfa: Generated new TSK 0
I20260808 07:15:11.599119 31450 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:15:11.631902 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.630258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41160 (local address 127.30.54.125:46533)
0808 07:15:11.630301 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.630304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.631190 (+ 886us) server_negotiation.cc:407] Connection header received
0808 07:15:11.631297 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.631299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.631311 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.631343 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.631831 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41160: BlockingRecv error: recv got EOF from 127.0.0.1:41160 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:15:11.632989 31401 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.631017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.30.54.124:36521)
0808 07:15:11.631059 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.631061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.632237 (+ 1176us) server_negotiation.cc:407] Connection header received
0808 07:15:11.632379 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.632380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.632390 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.632427 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.632924 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45964: BlockingRecv error: recv got EOF from 127.0.0.1:45964 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:15:11.633482 31202 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.630649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.30.54.126:44553)
0808 07:15:11.630692 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.630694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.632169 (+ 1475us) server_negotiation.cc:407] Connection header received
0808 07:15:11.632501 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.632502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.632515 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.632553 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.633420 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53862: BlockingRecv error: recv got EOF from 127.0.0.1:53862 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 07:15:11.635931 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.635201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.30.54.125:46533)
0808 07:15:11.635237 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.635238 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.635417 (+ 179us) server_negotiation.cc:407] Connection header received
0808 07:15:11.635523 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.635524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.635540 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.635569 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.635886 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41172: BlockingRecv error: recv got EOF from 127.0.0.1:41172 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:11.638975 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.638173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41188 (local address 127.30.54.125:46533)
0808 07:15:11.638211 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.638213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.638460 (+ 247us) server_negotiation.cc:407] Connection header received
0808 07:15:11.638559 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.638560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.638569 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.638597 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.638934 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41188: BlockingRecv error: recv got EOF from 127.0.0.1:41188 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:15:11.642730 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.641973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.30.54.125:46533)
0808 07:15:11.642007 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.642009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.642254 (+ 245us) server_negotiation.cc:407] Connection header received
0808 07:15:11.642356 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.642357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.642366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.642396 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.642695 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41200: BlockingRecv error: recv got EOF from 127.0.0.1:41200 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:11.648507 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.647724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.30.54.125:46533)
0808 07:15:11.647756 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.647758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.648021 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:15:11.648116 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.648117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.648126 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.648155 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.648467 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41204: BlockingRecv error: recv got EOF from 127.0.0.1:41204 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:15:11.658258 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.657520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.30.54.125:46533)
0808 07:15:11.657554 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.657556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.657778 (+ 222us) server_negotiation.cc:407] Connection header received
0808 07:15:11.657881 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.657882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.657891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.657918 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.658214 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41218: BlockingRecv error: recv got EOF from 127.0.0.1:41218 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 07:15:11.676271 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.675470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.30.54.125:46533)
0808 07:15:11.675514 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.675516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.675697 (+ 181us) server_negotiation.cc:407] Connection header received
0808 07:15:11.675807 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.675808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.675819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.675873 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.676213 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41234: BlockingRecv error: recv got EOF from 127.0.0.1:41234 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:15:11.710124 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.709382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41242 (local address 127.30.54.125:46533)
0808 07:15:11.709443 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.709445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.709576 (+ 131us) server_negotiation.cc:407] Connection header received
0808 07:15:11.709649 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.709651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.709663 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.709719 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.710050 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41242: BlockingRecv error: recv got EOF from 127.0.0.1:41242 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 07:15:11.776187 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.775354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41256 (local address 127.30.54.125:46533)
0808 07:15:11.775422 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.775424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.775539 (+ 115us) server_negotiation.cc:407] Connection header received
0808 07:15:11.775644 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.775645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.775660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.775729 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.776115 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41256: BlockingRecv error: recv got EOF from 127.0.0.1:41256 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 07:15:11.906315 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:11.905501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41258 (local address 127.30.54.125:46533)
0808 07:15:11.905567 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:11.905569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:11.905686 (+ 117us) server_negotiation.cc:407] Connection header received
0808 07:15:11.905722 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:11.905724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:11.905735 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:11.905795 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:11.906254 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41258: BlockingRecv error: recv got EOF from 127.0.0.1:41258 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 07:15:12.164486 31297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:12.163612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41260 (local address 127.30.54.125:46533)
0808 07:15:12.163704 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:12.163706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:12.163890 (+ 184us) server_negotiation.cc:407] Connection header received
0808 07:15:12.163981 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:12.163982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:12.163995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:12.164048 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:12.164397 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41260: BlockingRecv error: recv got EOF from 127.0.0.1:41260 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 07:15:12.499172 31438 catalog_manager.cc:1259] Loaded cluster ID: ecb665021ae246d68f4a9fd2544ae9c6
I20260808 07:15:12.499248 31438 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8: loading cluster ID for follower catalog manager: success
I20260808 07:15:12.500094 31438 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8: acquiring CA information for follower catalog manager: success
I20260808 07:15:12.500406 31438 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 83b3ec5e6f7541298cbc6ca7e6d1e8b8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:15:12.547686 31484 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4: acquiring CA information for follower catalog manager: success
I20260808 07:15:12.548069 31484 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ac96d1f70b50490593c814a7ec878df4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:15:12.680593 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:12.677713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41266 (local address 127.30.54.125:46533)
0808 07:15:12.677849 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:12.677851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:12.677885 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:15:12.678009 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:12.678010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:12.678069 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:12.678163 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:12.678319 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.678925 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.679692 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.679951 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.680028 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:12.680219 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:12.680357 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:12.680448 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:12.680475 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 07:15:12.680497 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260808 07:15:12.684513 31585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:12.682267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45980 (local address 127.30.54.124:36521)
0808 07:15:12.682383 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:12.682385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:12.682410 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:15:12.682506 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:12.682507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:12.682540 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:12.682597 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:12.682698 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.683171 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.683858 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.684038 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.684087 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:12.684211 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:12.684291 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:12.684343 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:12.684380 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:15:12.684395 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":47,"threads_started":1}
I20260808 07:15:12.688990 31585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:12.686028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45982 (local address 127.30.54.124:36521)
0808 07:15:12.686065 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:12.686067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:12.686249 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:15:12.686315 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:12.686316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:12.686326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:12.686379 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:12.686596 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.687169 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.688205 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.688489 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.688554 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:12.688711 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:12.688741 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:12.688791 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:12.688894 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:15:12.688915 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:12.689028 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:12.685966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41274 (local address 127.30.54.125:46533)
0808 07:15:12.686038 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:12.686039 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:12.686310 (+ 271us) server_negotiation.cc:407] Connection header received
0808 07:15:12.686438 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:12.686438 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:12.686450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:12.686508 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:12.686631 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.687350 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.688340 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.688579 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.688635 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:12.688787 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:12.688815 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:12.688862 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:12.688910 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 07:15:12.688927 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:15:12.689517 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:12.685985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.30.54.126:44553)
0808 07:15:12.686094 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:12.686096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:12.686268 (+ 172us) server_negotiation.cc:407] Connection header received
0808 07:15:12.686403 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:12.686404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:12.686456 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:12.686537 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:12.686709 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.687474 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.688261 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:12.688579 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:12.688615 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:12.688906 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:12.689001 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:12.689042 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:12.689379 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 07:15:12.689405 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":61,"threads_started":1}
I20260808 07:15:12.690868 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31097
I20260808 07:15:12.707320 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31206
I20260808 07:15:12.716544 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31301
2026-08-08T07:15:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3044 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:15:12 dist-test-slave-znh6 krb5kdc[31598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:15:12 dist-test-slave-znh6 krb5kdc[31598](info): set up 2 sockets
Aug 08 07:15:12 dist-test-slave-znh6 krb5kdc[31598](info): commencing operation
krb5kdc: starting...
W20260808 07:15:13.935935 30937 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/test-user.keytab.
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:15:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:15:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.54.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
Entry for principal kudu/127.30.54.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab.
I20260808 07:15:14.054239 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39033
--webserver_interface=127.30.54.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:39033,127.30.54.125:33077,127.30.54.124:46733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:15:14.134992 31616 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:14.135217 31616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:14.135267 31616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:14.135305 31616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:14.136408 31616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:14.136472 31616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:14.136497 31616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:14.136529 31616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:14.136559 31616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:14.138006 31616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:39033,127.30.54.125:33077,127.30.54.124:46733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.126.keytab
--principal=kudu/127.30.54.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.126:39033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.54.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:14.138280 31616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:14.138509 31616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:14.143384 31625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:14.143381 31627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:14.143378 31616 server_base.cc:1034] running on GCE node
W20260808 07:15:14.143429 31629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:14.143857 31616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:14.144153 31616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:14.145320 31616 hybrid_clock.cc:648] HybridClock initialized: now 1786173314145316 us; error 38 us; skew 500 ppm
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:14.152263 31616 init.cc:377] Logged in from keytab as kudu/127.30.54.126@KRBTEST.COM (short username kudu)
I20260808 07:15:14.152621 31616 webserver.cc:460] Webserver started at http://127.30.54.126:46429/ using document root <none> and password file <none>
I20260808 07:15:14.152786 31616 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:14.152825 31616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:14.152909 31616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:14.153846 31616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/data/instance:
uuid: "263c2ccfb98547ce91a94315f6fdc055"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-znh6"
server_key: "5679c84d2fa69399d19b77cecbfb10c4"
server_key_iv: "701972561b2475a2d6be8e00f108f9f1"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.154155 31616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/wal/instance:
uuid: "263c2ccfb98547ce91a94315f6fdc055"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-znh6"
server_key: "5679c84d2fa69399d19b77cecbfb10c4"
server_key_iv: "701972561b2475a2d6be8e00f108f9f1"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.155548 31616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:14.156283 31639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:14.156481 31616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:14.156574 31616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/wal
uuid: "263c2ccfb98547ce91a94315f6fdc055"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-znh6"
server_key: "5679c84d2fa69399d19b77cecbfb10c4"
server_key_iv: "701972561b2475a2d6be8e00f108f9f1"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.156639 31616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:14.166720 31616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:14.168426 31616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:14.168574 31616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:14.173441 31616 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.126:39033
I20260808 07:15:14.173516 31716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.126:39033 every 8 connection(s)
I20260808 07:15:14.173875 31616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/data/info.pb
I20260808 07:15:14.174847 31719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:14.176270 31719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:14.178903 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31616
I20260808 07:15:14.178982 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-0/wal/instance
I20260808 07:15:14.179168 30937 external_mini_cluster.cc:1428] Setting key 7c53e267058cb9b3fbb15de4e1d13aee
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:15:14.185830 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.176916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39033 (local address 127.0.0.1:53596)
0808 07:15:14.177103 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.177137 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.177558 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.177814 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.177816 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.178038 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.178185 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.178189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.179442 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.179443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.180103 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.180105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.180642 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.180643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.180705 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.181213 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:14.181221 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.182557 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.184357 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.184363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.184365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.184560 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.184810 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.184811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.184811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.184899 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.185252 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.185257 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.185315 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.185422 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.185663 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":108,"thread_start_us":92,"threads_started":1}
I20260808 07:15:14.185833 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.177090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53596 (local address 127.30.54.126:39033)
0808 07:15:14.177213 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.177217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.177253 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:15:14.177689 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.177690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.177706 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.177780 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.178277 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.179334 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.180209 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.180535 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.180652 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.182649 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.182656 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.182657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.182663 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.184259 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.184657 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.184659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.184660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.184743 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.184982 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.184983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.184983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.185180 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.185272 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.185463 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.185598 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":64,"threads_started":1}
I20260808 07:15:14.186589 31719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:14.186980 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.186803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:33077 (local address 127.0.0.1:47298)
0808 07:15:14.186901 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.186950 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260808 07:15:14.187134 31642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
I20260808 07:15:14.187579 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.187480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:33077 (local address 127.0.0.1:47314)
0808 07:15:14.187526 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.187556 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
WARNING: no policy specified for kudu/127.30.54.125@KRBTEST.COM; defaulting to no policy
W20260808 07:15:14.187734 31719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.125:33077: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab.
Entry for principal kudu/127.30.54.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab.
I20260808 07:15:14.205185 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.125:33077
--webserver_interface=127.30.54.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab
--principal=kudu/127.30.54.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:39033,127.30.54.125:33077,127.30.54.124:46733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:15:14.236980 31719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } attempt: 1
I20260808 07:15:14.237577 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.237408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:33077 (local address 127.0.0.1:47330)
0808 07:15:14.237501 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.237544 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 07:15:14.238019 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.237920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:33077 (local address 127.0.0.1:47344)
0808 07:15:14.237970 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.237998 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:15:14.238157 31719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.125:33077: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
W20260808 07:15:14.287019 31727 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:14.287209 31727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:14.287228 31727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:14.287248 31727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:14.288286 31727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:14.288322 31727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:14.288336 31727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:14.288349 31727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:14.288362 31727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:14.289812 31727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:39033,127.30.54.125:33077,127.30.54.124:46733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.125.keytab
--principal=kudu/127.30.54.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.125:33077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.54.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:14.290002 31727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:14.290158 31727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:14.295058 31740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:14.294978 31738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:14.294977 31736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:14.295006 31727 server_base.cc:1034] running on GCE node
I20260808 07:15:14.295526 31727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:14.295801 31727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:14.296969 31727 hybrid_clock.cc:648] HybridClock initialized: now 1786173314296966 us; error 40 us; skew 500 ppm
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:14.299563 31727 init.cc:377] Logged in from keytab as kudu/127.30.54.125@KRBTEST.COM (short username kudu)
I20260808 07:15:14.300024 31727 webserver.cc:460] Webserver started at http://127.30.54.125:43435/ using document root <none> and password file <none>
I20260808 07:15:14.300251 31727 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:14.300320 31727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:14.300479 31727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:14.301466 31727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/data/instance:
uuid: "08ef0f08a5384447984e98a05d0f71d3"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-znh6"
server_key: "406b08a4b864035c8dd5a99930bf28a4"
server_key_iv: "ad956f22267d19607b2634ea0d791a15"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.301820 31727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/wal/instance:
uuid: "08ef0f08a5384447984e98a05d0f71d3"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-znh6"
server_key: "406b08a4b864035c8dd5a99930bf28a4"
server_key_iv: "ad956f22267d19607b2634ea0d791a15"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.303149 31727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:15:14.303877 31749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:14.304046 31727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:15:14.304150 31727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/wal
uuid: "08ef0f08a5384447984e98a05d0f71d3"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-znh6"
server_key: "406b08a4b864035c8dd5a99930bf28a4"
server_key_iv: "ad956f22267d19607b2634ea0d791a15"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.304225 31727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:14.306336 31719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } attempt: 2
I20260808 07:15:14.306744 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.306609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:33077 (local address 127.0.0.1:47360)
0808 07:15:14.306673 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.306711 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:15:14.307096 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.306991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:33077 (local address 127.0.0.1:47372)
0808 07:15:14.307041 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.307070 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 07:15:14.307200 31719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.125:33077: Network error: Client connection negotiation failed: client connection to 127.30.54.125:33077: connect: Connection refused (error 111)
I20260808 07:15:14.327005 31727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:14.329181 31727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:14.329290 31727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:14.333166 31727 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.125:33077
I20260808 07:15:14.333199 31820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.125:33077 every 8 connection(s)
I20260808 07:15:14.333536 31727 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/data/info.pb
I20260808 07:15:14.334533 31821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:14.335950 31821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:14.339958 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31727
I20260808 07:15:14.340068 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-1/wal/instance
I20260808 07:15:14.340267 30937 external_mini_cluster.cc:1428] Setting key 6a41228e924e2976a7ff83b31a95028e
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:15:14.345017 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.336590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53600 (local address 127.30.54.126:39033)
0808 07:15:14.336651 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.336654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.336871 (+ 217us) server_negotiation.cc:407] Connection header received
0808 07:15:14.337304 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.337305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.337315 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.337368 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.337904 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.338373 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.339375 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.339577 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.339661 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.342157 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.342159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.342159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.342167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.343594 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.343991 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.343993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.343993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.344055 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.344324 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.344325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.344326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.344439 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.344516 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.344835 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.344917 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:15:14.345180 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.336571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39033 (local address 127.0.0.1:53600)
0808 07:15:14.336754 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.336790 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.337225 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.337433 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.337436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.337653 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.337834 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.337838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.338452 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.338452 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.339303 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.339306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.339682 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.339683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.339806 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.340301 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:14.340309 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.342038 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.343680 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.343688 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.343690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.343912 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.344124 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.344125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.344125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.344246 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.344574 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.344581 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.344699 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.344810 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.344959 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260808 07:15:14.345762 31821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.54.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.54.124@KRBTEST.COM" created.
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
I20260808 07:15:14.355194 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.346031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:33077 (local address 127.0.0.1:47380)
0808 07:15:14.346066 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.346077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.346120 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.346427 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.346429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.346575 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.346661 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.346663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.347987 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.347988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.348460 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.348462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.349101 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.349102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.349156 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.349491 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:14.349495 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.352341 (+ 2846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.354115 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.354116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.354116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.354282 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.354503 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.354504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.354504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.354558 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.354862 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.354862 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.354901 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.354967 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.355066 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 07:15:14.355284 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.346109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.30.54.125:33077)
0808 07:15:14.346270 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.346275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.346291 (+ 16us) server_negotiation.cc:407] Connection header received
0808 07:15:14.346317 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.346319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.346328 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.346388 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.346758 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.347892 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.348549 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.349024 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.349105 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.352475 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.352490 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.352491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.352496 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.354012 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.354375 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.354378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.354378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.354436 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.354629 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.354630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.354630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.354762 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.354866 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.355003 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.355135 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260808 07:15:14.356153 31821 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:14.356501 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.356360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39872)
0808 07:15:14.356412 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.356467 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:15:14.356653 31752 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
I20260808 07:15:14.357108 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.356988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39888)
0808 07:15:14.357044 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.357077 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:15:14.357225 31821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46733: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.54.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab.
Entry for principal kudu/127.30.54.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab.
I20260808 07:15:14.367064 30937 external_mini_cluster.cc:1326] Running /tmp/dist-test-task8VfA9O/build/release/bin/kudu
/tmp/dist-test-task8VfA9O/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.124:46733
--webserver_interface=127.30.54.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab
--principal=kudu/127.30.54.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.54.84:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.54.126:39033,127.30.54.125:33077,127.30.54.124:46733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:15:14.398352 31719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } attempt: 3
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
I20260808 07:15:14.407714 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.398751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:33077 (local address 127.0.0.1:47386)
0808 07:15:14.398832 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.398845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.398900 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.399081 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.399082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.399325 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.399433 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.399435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.400548 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.400549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.401573 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.401576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.402234 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.402236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.402343 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.402938 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:14.402973 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.404736 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.406660 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.406661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.406662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.406889 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.407109 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.407109 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.407110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.407163 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.407421 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.407422 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.407468 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.407520 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.407616 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:15:14.408231 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.398862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.30.54.125:33077)
0808 07:15:14.398938 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.398940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.398950 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:14.398977 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.398977 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.399008 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.399056 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.399529 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.400456 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.401696 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.402135 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.402197 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.404833 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.404835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.404835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.404842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.406570 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.406984 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.406985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.406986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.407040 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.407217 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.407218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.407218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.407333 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.407384 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.408070 (+ 686us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.408149 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:15:14.410877 31821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } attempt: 1
I20260808 07:15:14.411206 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.411103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39898)
0808 07:15:14.411145 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.411182 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 07:15:14.412071 31719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:14.412417 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.412308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39904)
0808 07:15:14.412350 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.412379 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 07:15:14.412585 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.412492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39920)
0808 07:15:14.412529 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.412559 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 07:15:14.412667 31821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46733: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
I20260808 07:15:14.412849 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.412689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39926)
0808 07:15:14.412798 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.412825 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260808 07:15:14.413000 31719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46733: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
I20260808 07:15:14.444134 31719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } attempt: 1
I20260808 07:15:14.444578 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.444461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39938)
0808 07:15:14.444521 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.444556 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:15:14.444974 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.444887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39952)
0808 07:15:14.444935 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.444959 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:15:14.445084 31719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46733: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
W20260808 07:15:14.453810 31829 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:15:14.454023 31829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:15:14.454082 31829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:15:14.454126 31829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:15:14.455296 31829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:15:14.455363 31829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:15:14.455400 31829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:15:14.455426 31829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:15:14.455456 31829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:15:14.456930 31829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.54.84:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.54.126:39033,127.30.54.125:33077,127.30.54.124:46733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/krb5kdc/kudu_127.30.54.124.keytab
--principal=kudu/127.30.54.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.54.124:46733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.54.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 05:42:04 UTC on ccycloud.quasar-fxrzxs.root.comops.site
build id 4574018
I20260808 07:15:14.457190 31829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:15:14.457418 31829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:15:14.462291 31841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:14.462334 31836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:15:14.462353 31837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:15:14.462488 31829 server_base.cc:1034] running on GCE node
I20260808 07:15:14.462711 31829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:15:14.463042 31829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:15:14.464183 31829 hybrid_clock.cc:648] HybridClock initialized: now 1786173314464166 us; error 42 us; skew 500 ppm
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:15:14.466531 31829 init.cc:377] Logged in from keytab as kudu/127.30.54.124@KRBTEST.COM (short username kudu)
I20260808 07:15:14.467057 31829 webserver.cc:460] Webserver started at http://127.30.54.124:33511/ using document root <none> and password file <none>
I20260808 07:15:14.467283 31829 fs_manager.cc:359] Metadata directory not provided
I20260808 07:15:14.467327 31829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:15:14.467486 31829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:15:14.468497 31829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/data/instance:
uuid: "844b596f76a6448fbaa91b0eb47b863a"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-znh6"
server_key: "2ee85d712d8d70d320e182d89afd4fdb"
server_key_iv: "ff70d404a6c11c6dc64cc09cdc088131"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.468848 31829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/wal/instance:
uuid: "844b596f76a6448fbaa91b0eb47b863a"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-znh6"
server_key: "2ee85d712d8d70d320e182d89afd4fdb"
server_key_iv: "ff70d404a6c11c6dc64cc09cdc088131"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.470182 31829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:15:14.471020 31850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:15:14.471218 31829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:15:14.471306 31829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/data,/tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/wal
uuid: "844b596f76a6448fbaa91b0eb47b863a"
format_stamp: "Formatted at 2026-08-08 07:15:14 on dist-test-slave-znh6"
server_key: "2ee85d712d8d70d320e182d89afd4fdb"
server_key_iv: "ff70d404a6c11c6dc64cc09cdc088131"
server_key_version: "encryptionkey@0"
I20260808 07:15:14.471359 31829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:15:14.480792 31821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } attempt: 2
I20260808 07:15:14.481207 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.481084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39968)
0808 07:15:14.481143 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.481180 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:15:14.481585 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.481497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39982)
0808 07:15:14.481540 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.481566 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:15:14.481688 31821 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.54.124:46733: Network error: Client connection negotiation failed: client connection to 127.30.54.124:46733: connect: Connection refused (error 111)
I20260808 07:15:14.483300 31829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:15:14.484946 31829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:15:14.485049 31829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:15:14.488771 31829 rpc_server.cc:307] RPC server started. Bound to: 127.30.54.124:46733
I20260808 07:15:14.488818 31918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.54.124:46733 every 8 connection(s)
I20260808 07:15:14.489130 31829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/data/info.pb
I20260808 07:15:14.490099 31919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:15:14.491582 31919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:15:14.492172 30937 external_mini_cluster.cc:1388] Started /tmp/dist-test-task8VfA9O/build/release/bin/kudu as pid 31829
I20260808 07:15:14.492288 30937 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task8VfA9O/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786173009519897-30937-0/minicluster-data/master-2/wal/instance
I20260808 07:15:14.492496 30937 external_mini_cluster.cc:1428] Setting key 04c2775b07a75af90acba8f2b0d765f1
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
I20260808 07:15:14.501073 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.492207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.30.54.126:39033)
0808 07:15:14.492264 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.492267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.492514 (+ 247us) server_negotiation.cc:407] Connection header received
0808 07:15:14.493220 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.493222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.493231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.493277 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.493816 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.494362 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.495516 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.495817 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.495898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.498089 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.498090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.498091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.498097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.499666 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.500107 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.500108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.500108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.500167 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.500444 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.500445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.500446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.500564 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.500636 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.500901 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.500983 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:15:14.501248 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.492178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.126:39033 (local address 127.0.0.1:53614)
0808 07:15:14.492386 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.492435 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.493103 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.493317 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.493321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.493591 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.493729 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.493733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.494439 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.494440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.495439 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.495441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.495942 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.495943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.496066 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.496558 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:14.496565 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.497992 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.499760 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.499766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.499768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.500018 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.500247 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.500248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.500249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.500365 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.500641 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.500648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.500770 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.500877 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.501033 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":110,"thread_start_us":110,"threads_started":1}
I20260808 07:15:14.501801 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.492913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39984 (local address 127.30.54.124:46733)
0808 07:15:14.493075 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.493079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.493116 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:15:14.493216 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.493218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.493245 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.493344 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.493984 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.495043 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.495906 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.496316 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.496368 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.498674 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.498681 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.498682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.498689 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.500286 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.500655 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.500658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.500658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.500715 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.500918 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.500919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.500919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.501057 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.501535 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.501601 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.501682 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260808 07:15:14.501900 31919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.126@KRBTEST.COM
I20260808 07:15:14.509500 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.502556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.30.54.126:39033)
0808 07:15:14.502594 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.502596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.502603 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:14.502622 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.502622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.502630 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.502680 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.503263 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.503754 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.504454 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.504668 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.504733 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.507112 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.507114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.507114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.507121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.508431 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.508769 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.508770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.508771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.508825 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.509032 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.509033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.509033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.509138 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.509295 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.509333 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.509401 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:15:14.509531 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.502301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.125:33077 (local address 127.0.0.1:47390)
0808 07:15:14.502332 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.502343 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.502541 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.502681 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.502682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.502861 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.502894 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.502895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.503632 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.503632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.504179 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.504181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.504617 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.504618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.504675 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.505068 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:14.505073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.506419 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.508373 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.508374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.508374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.508544 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.508767 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.508768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.508768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.508824 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.509200 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.509201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.509241 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.509327 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.509421 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260808 07:15:14.509626 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.502378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47390 (local address 127.30.54.125:33077)
0808 07:15:14.502445 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.502446 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.502472 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:15:14.502616 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.502617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.502626 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.502670 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.502975 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.503557 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.504256 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.504550 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.504600 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.506531 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.506532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.506533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.506539 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.508294 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.508632 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.508633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.508634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.508692 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.509017 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.509018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.509018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.509119 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.509195 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.509368 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.509441 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:15:14.510015 31919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.124@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.54.125@KRBTEST.COM
I20260808 07:15:14.516798 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.510262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:39992)
0808 07:15:14.510309 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.510319 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.510356 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.510522 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.510523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.510651 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.510673 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.510675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.511331 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.511332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.511782 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.511784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.512175 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.512176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.512224 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.512624 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:14.512629 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.513787 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.515785 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.515786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.515786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.515951 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.516154 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.516154 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.516155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.516237 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.516461 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.516462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.516510 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.516576 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.516677 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:15:14.516872 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.510317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39992 (local address 127.30.54.124:46733)
0808 07:15:14.510410 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.510412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.510418 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:15:14.510442 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.510443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.510451 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.510519 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.510744 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.511240 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.511881 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.512092 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.512174 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.513869 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.513870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.513871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.513877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.515705 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.516024 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.516025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.516025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.516084 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.516302 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.516302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.516303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.516391 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.516444 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.516622 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.516737 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":18}
I20260808 07:15:14.517601 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.510375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.30.54.125:33077)
0808 07:15:14.510421 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.510423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.510431 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:14.510457 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.510458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.510466 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.510515 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.511015 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.511445 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.512075 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.512287 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.512360 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.514445 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.514447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.514447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.514454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.516141 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.516524 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.516525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.516525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.516591 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.516804 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.516805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.516805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.516933 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.517087 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.517451 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.517526 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:15:14.518338 31919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a: Bootstrap starting.
I20260808 07:15:14.518977 30937 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:15:14.519392 31919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:14.519708 31919 log.cc:824] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:14.520244 31719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } attempt: 2
I20260808 07:15:14.520478 31919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a: No bootstrap required, opened a new log
I20260808 07:15:14.520160 31675 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "263c2ccfb98547ce91a94315f6fdc055"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53620
I20260808 07:15:14.522384 31919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } }
I20260808 07:15:14.522526 31919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 844b596f76a6448fbaa91b0eb47b863a, State: Initialized, Role: FOLLOWER
I20260808 07:15:14.522598 31919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } }
I20260808 07:15:14.522891 31919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:14.522877 31928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.523006 31928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:15:14.524765 31941 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:14.524821 31941 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.126@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
I20260808 07:15:14.524971 31829 master_runner.cc:186] Error getting master registration for 127.30.54.126:39033: OK, instance_id { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" instance_seqno: 1786173314171929 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:15:14.527884 31829 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:15:14.530640 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.520501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40002 (local address 127.30.54.124:46733)
0808 07:15:14.522926 (+ 2425us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.522927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.522936 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:15:14.522953 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.522954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.522964 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.523341 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.523368 (+ 27us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.524040 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.524969 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.525411 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.525745 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.527632 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.527634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.527634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.527641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.529420 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.529832 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.529833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.529833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.529888 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.530102 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.530103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.530103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.530247 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.530311 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.530477 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.530554 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2408}
I20260808 07:15:14.530639 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.520465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:40002)
0808 07:15:14.520508 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.520517 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.520553 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.523063 (+ 2510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.523064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.523221 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.523273 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.523275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.524102 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.524103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.524834 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.524836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.525524 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.525525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.525594 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.526009 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:14.526016 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.527534 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.529511 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.529512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.529512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.529752 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.529959 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.529960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.529961 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.530033 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.530320 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.530321 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.530368 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.530453 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.530548 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 07:15:14.532127 31719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055: Bootstrap starting.
I20260808 07:15:14.532788 31719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:14.533078 31719 log.cc:824] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:14.533767 31719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055: No bootstrap required, opened a new log
I20260808 07:15:14.534722 31719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } }
I20260808 07:15:14.534883 31719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263c2ccfb98547ce91a94315f6fdc055, State: Initialized, Role: FOLLOWER
I20260808 07:15:14.534984 31719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } }
I20260808 07:15:14.535287 31946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.535337 31675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:15:14.535321 31719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:14.535389 31946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:14.535427 31675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:14.536252 31675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } }
I20260808 07:15:14.536496 31675 leader_election.cc:288] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [CANDIDATE]: Term 1 election: Requested vote from peers 08ef0f08a5384447984e98a05d0f71d3 (127.30.54.125:33077), 844b596f76a6448fbaa91b0eb47b863a (127.30.54.124:46733)
I20260808 07:15:14.536790 31884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "263c2ccfb98547ce91a94315f6fdc055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "844b596f76a6448fbaa91b0eb47b863a"
I20260808 07:15:14.536916 31884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [term 0 FOLLOWER]: Advancing to term 1
W20260808 07:15:14.537293 31958 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:14.537340 31958 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:14.537815 31884 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 263c2ccfb98547ce91a94315f6fdc055 in term 1.
I20260808 07:15:14.538014 31642 leader_election.cc:302] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 263c2ccfb98547ce91a94315f6fdc055, 844b596f76a6448fbaa91b0eb47b863a; no voters:
I20260808 07:15:14.538147 31946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:15:14.538306 31946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [term 1 LEADER]: Becoming Leader. State: Replica: 263c2ccfb98547ce91a94315f6fdc055, State: Running, Role: LEADER
I20260808 07:15:14.538389 31946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } }
I20260808 07:15:14.538775 31946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 263c2ccfb98547ce91a94315f6fdc055. Latest consensus state: current_term: 1 leader_uuid: "263c2ccfb98547ce91a94315f6fdc055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.538916 31946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:14.538187 31616 master_runner.cc:186] Error getting master registration for 127.30.54.125:33077: OK, instance_id { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" instance_seqno: 1786173314332048 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:15:14.541021 31616 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:15:14.546916 31975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:15:14.547384 31975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:15:14.550174 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.538085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.30.54.126:39033)
0808 07:15:14.538143 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.538144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.541436 (+ 3292us) server_negotiation.cc:407] Connection header received
0808 07:15:14.541542 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.541543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.541567 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.541615 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.542924 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.543622 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.544643 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.544942 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.545006 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.547272 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.547273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.547274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.547281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.548892 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.549258 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.549259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.549259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.549310 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.549478 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.549479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.549480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.549604 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.549704 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.550017 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.550087 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:15:14.551225 31884 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [term 1 FOLLOWER]: Refusing update from remote peer 263c2ccfb98547ce91a94315f6fdc055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:14.551613 31946 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:15:14.552335 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.541164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.30.54.125:33077)
0808 07:15:14.541213 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.541215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.541310 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:15:14.541384 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.541385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.541395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.541474 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.542225 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.543003 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.545520 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.546146 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.546211 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.548778 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.548780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.548780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.548786 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.550545 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.551346 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.551347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.551347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.551409 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.551632 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.551633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.551634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.551783 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.551869 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.552103 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.552186 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:15:14.552920 31928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 263c2ccfb98547ce91a94315f6fdc055. Latest consensus state: current_term: 1 leader_uuid: "263c2ccfb98547ce91a94315f6fdc055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.553001 31928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:14.554668 31946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "263c2ccfb98547ce91a94315f6fdc055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.554746 31946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:14.555255 31928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "263c2ccfb98547ce91a94315f6fdc055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.555331 31928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:14.555716 31979 mvcc.cc:205] Tried to move back new op lower bound from 7316165896399167488 to 7316165896349962240. Current Snapshot: MvccSnapshot[applied={T|T < 7316165896399167488}]
I20260808 07:15:14.556219 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.539348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40008 (local address 127.30.54.124:46733)
0808 07:15:14.539380 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.539382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.547225 (+ 7843us) server_negotiation.cc:407] Connection header received
0808 07:15:14.547321 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.547322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.547331 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.547360 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.548810 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.549508 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.550570 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.550862 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.550910 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.552831 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.552832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.552832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.552839 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.554387 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.554976 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.554977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.554978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.555049 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.555354 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.555355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.555355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.555486 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.555565 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.556004 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.556081 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:15:14.556010 31980 mvcc.cc:205] Tried to move back new op lower bound from 7316165896399167488 to 7316165896349962240. Current Snapshot: MvccSnapshot[applied={T|T < 7316165896399167488}]
I20260808 07:15:14.563220 31975 catalog_manager.cc:1347] Generated new cluster ID: 4fae0a6e6b7848ab88b70b085a5f655f
I20260808 07:15:14.563287 31975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:15:14.572808 31821 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } attempt: 3
Aug 08 07:15:14 dist-test-slave-znh6 krb5kdc[31598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173314, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.54.125@KRBTEST.COM for kudu/127.30.54.124@KRBTEST.COM
I20260808 07:15:14.579689 31975 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:15:14.580927 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.573061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.54.124:46733 (local address 127.0.0.1:40014)
0808 07:15:14.573108 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:15:14.573118 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:15:14.573153 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:15:14.573344 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:15:14.573345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:15:14.573524 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:15:14.573582 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.573584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.574493 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.574494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.575328 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:15:14.575331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.575884 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.575885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:15:14.575940 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.576382 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:15:14.576387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:15:14.577945 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:15:14.579858 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.579859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.579859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.580083 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.580287 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:15:14.580288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:15:14.580288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:15:14.580343 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:15:14.580621 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:15:14.580622 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:15:14.580663 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 07:15:14.580727 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:15:14.580820 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:15:14.581314 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:14.573112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40014 (local address 127.30.54.124:46733)
0808 07:15:14.573213 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.573215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.573222 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:14.573239 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.573240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.573249 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.573298 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.573673 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.574393 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.575474 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:14.575778 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:14.575835 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:14.578055 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:15:14.578056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:15:14.578057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:15:14.578063 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:15:14.579765 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.580166 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.580167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.580167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.580222 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:15:14.580411 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:15:14.580412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:15:14.580413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:15:14.580535 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:15:14.580594 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:14.581152 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 07:15:14.581225 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 07:15:14.581712 31975 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:15:14.582762 31821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3: Bootstrap starting.
I20260808 07:15:14.583508 31821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3: Neither blocks nor log segments found. Creating new log.
I20260808 07:15:14.583801 31821 log.cc:824] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:15:14.584859 31821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3: No bootstrap required, opened a new log
I20260808 07:15:14.586874 31821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } }
I20260808 07:15:14.587033 31821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08ef0f08a5384447984e98a05d0f71d3, State: Initialized, Role: FOLLOWER
I20260808 07:15:14.587113 31821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } }
I20260808 07:15:14.587417 31821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:15:14.587464 31783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "263c2ccfb98547ce91a94315f6fdc055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "08ef0f08a5384447984e98a05d0f71d3"
I20260808 07:15:14.587577 31783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:15:14.588371 31783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 263c2ccfb98547ce91a94315f6fdc055 in term 1.
I20260808 07:15:14.588577 31782 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [term 1 FOLLOWER]: Refusing update from remote peer 263c2ccfb98547ce91a94315f6fdc055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:15:14.588974 31968 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:15:14.589382 32003 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:15:14.589437 32003 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:15:14.591156 32008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 263c2ccfb98547ce91a94315f6fdc055. Latest consensus state: current_term: 1 leader_uuid: "263c2ccfb98547ce91a94315f6fdc055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.591288 32008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:14.591377 32008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "263c2ccfb98547ce91a94315f6fdc055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.591729 32008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:14.591995 31990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "263c2ccfb98547ce91a94315f6fdc055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.592090 31990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:15:14.592949 32006 mvcc.cc:205] Tried to move back new op lower bound from 7316165896503754752 to 7316165896349962240. Current Snapshot: MvccSnapshot[applied={T|T < 7316165896399167488}]
I20260808 07:15:14.599267 31946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "263c2ccfb98547ce91a94315f6fdc055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "263c2ccfb98547ce91a94315f6fdc055" member_type: VOTER last_known_addr { host: "127.30.54.126" port: 39033 } } peers { permanent_uuid: "08ef0f08a5384447984e98a05d0f71d3" member_type: VOTER last_known_addr { host: "127.30.54.125" port: 33077 } } peers { permanent_uuid: "844b596f76a6448fbaa91b0eb47b863a" member_type: VOTER last_known_addr { host: "127.30.54.124" port: 46733 } } }
I20260808 07:15:14.599355 31946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055 [sys.catalog]: This master's current role is: LEADER
I20260808 07:15:14.600216 31975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 263c2ccfb98547ce91a94315f6fdc055: Generated new TSK 0
I20260808 07:15:14.600371 31975 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:15:14.631693 31723 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.627330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.30.54.126:39033)
0808 07:15:14.627367 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.627369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.631015 (+ 3646us) server_negotiation.cc:407] Connection header received
0808 07:15:14.631130 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.631131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.631143 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.631171 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.631625 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53640: BlockingRecv error: recv got EOF from 127.0.0.1:53640 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:15:14.632481 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.627133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.30.54.125:33077)
0808 07:15:14.627177 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.627179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.631950 (+ 4771us) server_negotiation.cc:407] Connection header received
0808 07:15:14.632060 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.632061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.632073 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.632103 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.632420 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47426: BlockingRecv error: recv got EOF from 127.0.0.1:47426 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:15:14.633184 31925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.627770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40026 (local address 127.30.54.124:46733)
0808 07:15:14.627805 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.627807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.632707 (+ 4900us) server_negotiation.cc:407] Connection header received
0808 07:15:14.632796 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.632797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.632807 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.632836 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.633126 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40026: BlockingRecv error: recv got EOF from 127.0.0.1:40026 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:15:14.635787 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.634711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47430 (local address 127.30.54.125:33077)
0808 07:15:14.634747 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.634749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.635277 (+ 528us) server_negotiation.cc:407] Connection header received
0808 07:15:14.635381 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.635382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.635407 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.635436 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.635747 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47430: BlockingRecv error: recv got EOF from 127.0.0.1:47430 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:14.638742 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.637959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47446 (local address 127.30.54.125:33077)
0808 07:15:14.637993 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.637995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.638237 (+ 242us) server_negotiation.cc:407] Connection header received
0808 07:15:14.638335 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.638336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.638347 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.638376 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.638705 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47446: BlockingRecv error: recv got EOF from 127.0.0.1:47446 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:14.642642 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.641900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47460 (local address 127.30.54.125:33077)
0808 07:15:14.641934 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.641935 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.642159 (+ 224us) server_negotiation.cc:407] Connection header received
0808 07:15:14.642252 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.642253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.642262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.642291 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.642602 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47460: BlockingRecv error: recv got EOF from 127.0.0.1:47460 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:15:14.648605 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.647816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.30.54.125:33077)
0808 07:15:14.647850 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.647851 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.648095 (+ 244us) server_negotiation.cc:407] Connection header received
0808 07:15:14.648192 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.648193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.648202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.648231 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.648564 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47468: BlockingRecv error: recv got EOF from 127.0.0.1:47468 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:15:14.658507 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.657710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47480 (local address 127.30.54.125:33077)
0808 07:15:14.657745 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.657746 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.657993 (+ 247us) server_negotiation.cc:407] Connection header received
0808 07:15:14.658089 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.658090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.658100 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.658129 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.658466 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47480: BlockingRecv error: recv got EOF from 127.0.0.1:47480 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:15:14.676662 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.675741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47494 (local address 127.30.54.125:33077)
0808 07:15:14.675782 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.675784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.676056 (+ 272us) server_negotiation.cc:407] Connection header received
0808 07:15:14.676162 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.676164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.676175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.676205 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.676596 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47494: BlockingRecv error: recv got EOF from 127.0.0.1:47494 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:15:14.710733 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.709977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47504 (local address 127.30.54.125:33077)
0808 07:15:14.710029 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.710031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.710181 (+ 150us) server_negotiation.cc:407] Connection header received
0808 07:15:14.710258 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.710259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.710270 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.710316 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.710665 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47504: BlockingRecv error: recv got EOF from 127.0.0.1:47504 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 07:15:14.776664 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.775877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.30.54.125:33077)
0808 07:15:14.775966 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.775968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.776067 (+ 99us) server_negotiation.cc:407] Connection header received
0808 07:15:14.776147 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.776148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.776160 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.776239 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.776600 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47512: BlockingRecv error: recv got EOF from 127.0.0.1:47512 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 07:15:14.906502 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:14.905835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.30.54.125:33077)
0808 07:15:14.905925 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:14.905927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:14.905987 (+ 60us) server_negotiation.cc:407] Connection header received
0808 07:15:14.906046 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:14.906047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:14.906059 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:14.906112 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:14.906440 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47514: BlockingRecv error: recv got EOF from 127.0.0.1:47514 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 07:15:15.164482 31826 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:15:15.163833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.30.54.125:33077)
0808 07:15:15.163936 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:15.163938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:15.163949 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:15:15.163995 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:15.163996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:15.164013 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:15.164077 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:15:15.164428 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47522: BlockingRecv error: recv got EOF from 127.0.0.1:47522 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 07:15:15.525214 31941 catalog_manager.cc:1259] Loaded cluster ID: 4fae0a6e6b7848ab88b70b085a5f655f
I20260808 07:15:15.525285 31941 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a: loading cluster ID for follower catalog manager: success
I20260808 07:15:15.526046 31941 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a: acquiring CA information for follower catalog manager: success
I20260808 07:15:15.526292 31941 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 844b596f76a6448fbaa91b0eb47b863a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:15:15.589835 32003 catalog_manager.cc:1259] Loaded cluster ID: 4fae0a6e6b7848ab88b70b085a5f655f
I20260808 07:15:15.589931 32003 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3: loading cluster ID for follower catalog manager: success
I20260808 07:15:15.590737 32003 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3: acquiring CA information for follower catalog manager: success
I20260808 07:15:15.591094 32003 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 08ef0f08a5384447984e98a05d0f71d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:15:15.680544 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:15.677704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.30.54.125:33077)
0808 07:15:15.677843 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:15.677845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:15.677871 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:15:15.677969 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:15.677970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:15.678050 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:15.678142 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:15.678311 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.678859 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.679660 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.679886 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.679979 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:15.680177 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:15.680296 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:15.680384 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:15.680425 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 07:15:15.680449 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":57,"threads_started":1}
I20260808 07:15:15.684712 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:15.682383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.30.54.124:46733)
0808 07:15:15.682505 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:15.682507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:15.682514 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:15:15.682549 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:15.682550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:15.682581 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:15.682646 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:15.682729 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.683264 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.683946 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.684132 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.684184 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:15.684375 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:15.684475 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:15.684531 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:15.684581 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 07:15:15.684599 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260808 07:15:15.689291 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:15.686402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60640 (local address 127.30.54.126:39033)
0808 07:15:15.686710 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:15.686713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:15.686723 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:15:15.686776 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:15.686777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:15.686852 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:15.686971 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:15.687095 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.687593 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.688449 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.688655 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.688731 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:15.688925 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:15.688992 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:15.689060 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:15.689131 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:15:15.689154 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":60,"threads_started":1}
I20260808 07:15:15.689587 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:15.686440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.30.54.125:33077)
0808 07:15:15.686480 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:15.686482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:15.686573 (+ 91us) server_negotiation.cc:407] Connection header received
0808 07:15:15.686674 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:15.686675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:15.686685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:15.686716 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:15.686893 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.687618 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.688751 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.689033 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.689070 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:15.689308 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:15.689342 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:15.689372 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:15.689495 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 07:15:15.689516 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:15:15.689615 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:15:15.686680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.30.54.124:46733)
0808 07:15:15.686713 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:15:15.686715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:15:15.686723 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:15:15.686776 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:15:15.686777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:15:15.686787 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:15:15.686883 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:15:15.687135 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.687797 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.688818 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:15:15.689118 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:15:15.689154 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:15:15.689331 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:15:15.689368 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:15:15.689423 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:15:15.689522 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:15:15.689547 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:15:15.691159 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31616
I20260808 07:15:15.705574 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31727
I20260808 07:15:15.714012 30937 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task8VfA9O/build/release/bin/kudu with pid 31829
2026-08-08T07:15:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2996 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6040 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (306204 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS